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FOREWORD 


The  SURVEILLANCE  SYSTEMS  research  program  of  the  U.  S.  Army  Behavioral  Science 
Research  Laboratory  has  as  its  objective  the  production  of  scientific  data  tearing  on  the 
extraction  of  information  from  surveillance  displays,  and  the  efficient  storage,  retrieval, 
and  transmission  of  this  information  within  an  advanced  computerized  image  interpreta¬ 
tion  facility.  Research  results  are  used  in  future  systems  design  and  in  the  development 
of  enhanced  techniques  for  all  phases  of  the  interpretation  process.  Research  is  con¬ 
ducted  under  Army  RDT&E  Project  2Q662704A721,  “Surveillance  Systems,”  and 
2Q662704A732,  “linage  Characteristics  and  Interpreter  Performance,"  FY  1970  Work 
Program. 

The  present  publication  presents  the  major  problem  areas,  the  rationale  of  BESRL's 
approach  to  their  solution,  and  the  general  course  of  research  studies  completed  or  in 
progress  in  each  area  of  related  problems.  The  research  effort  is  currently  organized 
into  the  following  Work  Units: 

The  Determination  of  Interpreter  Techniques  in  a  Surveillance  Facility— 

INTERPRETER  TECHNIQUES 

Influence  of  Displays  on  Image  Interpreter  Performance-IMAGE  INTERPRE¬ 
TATION  DISPLAYS 

Intelligence  Information  Processing  Systems-INTELLIGENCE  SYSTEMS 

Information  Processing  in  Advanced  Image  Interpretation  Systems-IMAGE 

SYSTEMS 

BESRL research  in  surveillance  systems  is  conducted  as  an  in-house  effort  augmented 
by  contracts  with  organizations  selected  as  having  unique  capabilities  and  facilities  for 
research  in  aerial  surveillance.  The  experimentation  is  in  the  main  performed  in  BESRL's 
Information  Systems  Laboratory. 
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BRIEF 


Requirements: 

INTERPRETER  TECHNIQUES.  To  develop  methods  and  procedures  which  maximize  the 
accuracy,  completeness,  and  speed  with  which  intelligence  information  is  derived  from 
imagery-photographic,  infrared,  and  radar. 

IMAGE  INTERPRETATION  DISPLAYS.  To  determine  how  interpreter  performarce  is 
affected  by  variations  in  the  characteristics  of  the  image-magnification  and  image 
quality  of  photos,  for  example,  and  output  of  infrared  and  radar  sensors;  to  specify 
techniques  for  accurate  and  speedy  reporting  of  nformation;  and  to  select  or  develop 
efficient  procedures  for  change  detection  in  comparing  early  and  late  cover. 

IMAGE  SYSTEMS.  To  integrate,  evaluate,  and  improve  advanced  surveillance  informa¬ 
tion  processing  systems  through  laboratory  simulation  and  to  develop  effective  tech¬ 
niques  for  team  operations,  data  bank  utilization,  and  control  of  imagery  and  information 
flow  through  the  system. 

INTELLIGENCE  SYSTEMS.  To  increase  the  speed,  accuracy,  and  completeness  of 
field  army  intelligence  processing  in  advanced  computerized  systems  through  research 
on  man/machine  functions,  procedures,  and  information  management. 


Scope  of  the  Present  Publication: 

The  present  Technical  Research  Report  summarizes  in  integrated  fashion  the  ratio¬ 
nale,  broad  objectives,  and  specific  studies  of  the  Surveillance  Systems  research  pro¬ 
grams  conducted  by  the  Support  Systems  Research  Division  of  BESRL  and  delineates  the 
major  areas  in  which  BESRL  s  manned  systems  experimentation  has  resulted  in  findings 
of  interest  to  the  Office,  Chief  of  Research  and  Development,  the  Assistant  Chief  of 
Staff  for  Intelligence,  the  Assistant  Chief  of  Staff  for  Force  Development,  and  the  U.  S. 
Continental  Army  Command.  Findings  are  applicable  in  optimizing  performance  of  the 
human  component  in  existing  systems  and  in  providing  systems  developers  with  informa¬ 
tion  useful  in  design  specifications  for  future  systems. 
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BACKGROUND 

Military  decisions  require  an  estimate  of  the  enemy's  strength  and 
weaknesses,  his  manner  of  deployment,  and  his  possible  courses  of  action. 
To  provide  the  needed  information,  many  sources  of  intelligence  must  be 
utilized.  One  major  source  is  aerial  surveillance. 

A  significant  intermediary  output  of  aerial  surveillance  systems  is 
the  imagery  generated  by  the  various  sensor  systems  flown  to  cover  the 
terrain  of  interest.  The  image  interpreter  is  the  Individual  who  trans¬ 
forms  the  information  recorded  on  these  images  into  intelligence  infor¬ 
mation.  Not  too  long  ago,  the  interpreter's  problem  was  restricted  to 
the  interpretation  of  black  and  white  photographs.  Nowadays,  he  must 
also  extract  information  from  images  generated  by  the  more  exotic  sensors 
such  as  infrared  and  radar,  and  what  is  more,  the  information  must  be 
produced  rapidly.  Moreover,  with  the  ever-increasing  volume  of  imagery 
the  new  systems  are  able  to  generate,  interpretation  facilities  are 
being  inundated.  Within  this  context,  the  focus  is  on  the  interpreter 
who  must  provide  intelligence  information  meeting  standards  of  accept¬ 
ability.  Based  on  the  current  state  of  the  art,  it  is  expected  that  he 
will  remain  the  key  information  transformation  agent  for  some  time  to 
come. 


"intelligence"  is  information  and  as  such  possesses  characteristics 
of  information.  From  the  usability  point  of  view,  there  are  two  majcr 
aspects  of  concern:  the  information  content  and  certain  information 
attributes.  The  intelligence  content  is  important  because  it  relates 
directly  to  the  requirements  imposed  on  the  surveillance  system  and 
directly  on  the  interpreter.  The  information  attributes  provide  the 
means  for  direct  empirical  assessment  of  the  information  provided  and 
thus  can  serve  as  the  yardsticks  for  many  comparisons.  BESRL 's  manned 
systems  research  rests  on  the  measurement  oi  these  attributes--the 
accuracy  and  completeness  of  information  and  the  speed  with  which  it  is 
generated. 

Research  in  the  area  of  surveillance  systems  must  take  into  account 
a  multiplicity  of  variables.  There  are,  in  fact,  families  of  variables. 
For  example,  with  respect  to  imagery,  there  is  concern  not  only  with  type 
of  image--photo,  infrared,  and  radar--but  in  photo  imagery  alone,  there 
are  vertical,  oblique,  and  panoramic  views.  Photographs  differ  in  scale 
and  quality  dimensions.  They  differ  in  content:  They  may  depict  differ¬ 
ent  terrain  (jungle,  mountainous,  swamp,  desert);  they  may  have  been 
obtained  under  different  weather  conditions  (cloud  cover,  snow,  rain, 
sunny);  they  may  show  different  degrees  of  enemy  deployment  in  terms  of 
target  type  and  target  numerosity.  With  respect  to  the  interpreter, 
there  are  differences  in  ability,  background,  experience,  and  training. 

In  requirements,  there  are  differences  as  a  function  of  echelon;  for 
example,  the  commander  of  an  army  is  more  concerned  with  the  global 


picture  than  with  knowing  where  the  foxholes  are.  The  company  commander, 
for  hi®  part,  is  vitally  concerned  with  where  enemy  obstacles  are  located, 
Within  echelon  there  are  different  information  requirements.  The  artil¬ 
lery  officer  is  concerned  with  targeting  information  and  therefore  needs 
map  coordinate  data  of  sufficient  precision  that  his  artillery  fire  will 
include  the  target  within  the  effective  radius  of  the  shell  bursts.  The 
engineering  officer  may  wish  to  locate  the  best  place  to  cross  the  river 
and  to  know  the  width  of  the  river  in  order  to  plan  for  the  requisite 
number  of  pontoons  to  span  the  river.  The  infantry  officer  whose  forces 
are  attacking  may  be  concerned  with  the  location  of  enemy  ambush  sites, 
their  mine  fields,  etc.;  in  the  defensive  posture,  he  may  need  to  know 
what  kind  of  enemy  forces  will  attack  him  and  where  they  are  most  likely 
to  attack.  Superimposed  on  these  requirements  are  the  requirements  that 
the  information  provided  be  accurate  and  complete  and  that  it  be  provided 
"yesterday. "  There  are  hosts  of  other  factors  that  enter  into  the 
picture,  such  as  display,  equipment,  and  procedures. 

A  surveillance  system  built  for  the  Army  should  be  able  to  provide 
for  the  needs  of  all  its  users.  With  the  multiplicity  of  variables  that 
must  be  expected  to  affect  performance,  system  developers  are  confronted 
with  numerous  trade-off  situations  where  decisions  must  be  made  whether 
to  go  one  way  or  another.  Data  are  desperately  needed  to  assist  in 
coming  to  logical  conclusions. 

In  the  area  of  intelligence  information,  BESRL  research  deals  with 
at  least  three  performance  indices — accuracy,  completeness,  and  timeli¬ 
ness  of  information.  Results  in  terms  of  these  indices  are  not  always 
in  the  same  direction.  For  example,  BESRL  studies  indicate  that,  while 
cumulative  completeness  increases  as  a  function  of  time,  cumulative 
accuracy  decreases.  Furthermore,  accuracy  can  be  emphasized  at  the 
expense  of  completeness  and  vice  versa.  There  are  various  other  trade¬ 
offs  involving  accuracy,  completeness,  and  time  that  can  be  made.  The 
problem  concerning  the  man-machine  interface  resides  in  the  nature  of 
responses  interpreters  can  and  do  make  when  they  interpret  an  image. 

The  interpreter  may  provide  a  number  of  correct  detections  and  identifi¬ 
cations.  However,  he  may  also  fail  to  detect  targets  actually  present, 
he  may  misidentify  others,  and  under  some  conditions  he  may  report 
targets  that  are  not  present.  In  other  words,  the  information  generated 
by  interpreters  possesses  an  attribute  of  fallibility. 

The  research  described  in  the  present  report  is  directed  toward  the 
improvement  of  information  output  of  future  surveillance,  interpretation, 
and  intelligence  systems  with  fallout  for  current  systems.  Studies  and 
findings  are  of  interest  to  the  users  of  these  systems  (interpreters,  G2 
and  G2  Air,  commanders,  etc.),  system  designers  and  engineers,  the  Intel¬ 
ligence  School,  and  the  intelligence  community  at  large. 


-  2  - 


Approach  in  BESRL's  Surveillance  Systems  Research 

The  systems  concept  played  a  major  role  in  the  formulation  of  the 
research  approach  in  the  area  of  surveillance.  A  system  can  be  thought 
of  In  terms  of  structure  and  function.  It  consists  of  personnel ,  equip- 
men' ,  materials,  procedures,  and  environmental  conditions.  It  receives 
Input,  performs  operations,  and  provides  output. 

A  system  can  also  be  thought  of  as  consisting  of  subsystems,  each 
with  its  own  input,  operations,  and  output,  and  each  funneling  its  out¬ 
put  into  the  larger  system.  System  improvement  can  be  achieved  through 
the  improvement  of  its  subsystems  and  components.  Assessment  of  sub¬ 
system  effectiveness  can  be  based  on  output  measures  of  two  types: 

1)  those  based  on  output  of  the  subsystems  and  2)  those  based  on  total 
system  output.  The  ultimate  determiner  of  subsystem  effectiveness  is, 
of  course,  its  contribution  to  the  total  system.  However,  subsystem 
output  may  be  useful  in  itself  in  many  studies,  and  at  times  may  be  the 
only  measurable  output  available  for  assessment. 

BESRL  studies  typically  fall  into  two  corresponding  categories, 
those  concerned  with  assessing  subsystems  or  subsystem  components  as 
entities  in  themselves,  and  those  concerned  with  the  integration  of  sub¬ 
systems  or  components  into  a  larger  system  for  total  system  evaluation. 
The  approach  has  been  to  analyze  current  systems  and  conceptualizations 
of  future  systems  for  the  purpose  of  study  formulation  and  execution. 

For  either  category  of  study,  the  research  is  planned  so  as  to  yield 
empirical  performance  data  reflective  of  the  maximum  amount  of  opera¬ 
tional  realism  compatible  with  appropriate  experimental  control.  Most 
studies  have  as  a  basic  sequence  the  presentation  of  tactical  imagery 
to  trained  Army  image  interpreters  and  the  requirement  that  they  perform 
their  usual  tasks  of  detection,  identification,  and  mensuration.  Their 
responses  are  then  scored  for  accuracy  (percent  of  responses  correct), 
completeness  (percent  of  targets  actually  present  in  the  imagery  that 
are  reported),  and  speed  (number  of  targets  or  reports  per  unit  of  time). 
The  studies  vary  widely,  depending  on  the  number,  type,  and  complexity 
of  the  variables  being  studied. 

In  the  simplest  case,  little  may  be  required  beyond  use  of  a  light 
table  and  the  assortment  of  manual  aids  found  in  the  standard  photo¬ 
interpreter  kit — tube  magnifiers,  slide  rule,  etc.  The  more  complex 
studies  deal  with  a  full  scale  semi-automated  computerized  imagery 
interpretation  system.  Interpreter,  equipment,  and  procedures  are 
totally  interrelated.  Any  change  in  an  element  of  such  a  system  affects 
other  elements  in  the  system--not  necessarily  in  the  desired  or  expected 
direction. 


BESRL's  Information  Systems  Laboratory 

To  accomplish  research  in  a  timely  and  meaningful  manner  for  such 
complex  systems,  which  frequently  are  still  in  the  conceptualization 
stage,  an  Information  Systems  Laboratory  has  been  constructed  within  the 
Support  Systems  Research  Division  of  BGSRL.  This  laboratory  contains 
light  tables,  full  frame  magnification  viewers,  random  access  slide  pro¬ 
jectors,  cathode  ray  tubes  with  keyboards,  pushbutton  matrices,  and  type¬ 
writer  keyboards.  More  important,  all  equipment  is  on-line  to  BESRL's 
CDC  3300  computer*-' which  is  time-shared  with  the  normal  statistical 
processing  of  research  data.  The  time-sharing  is  on  a  priority-interrupt 
basis,  the  laboratory  experimentation  having  priority.  A  detailed  de¬ 
scription  of  the  laboratory  and  its  operation  is  presented  as  Appendix. A. 

A  prime  virtue  of  the  Information  Systems  Laboratory  is  its  flexi¬ 
bility.  Not  only  does  it  permit  empirical  simulation  of  computerized 
systems  which  could  not  otherwise  be  studied,  but  it  also  facilitates 
experimental  control  and  data  collection  for  studies  where  computeriza¬ 
tion  is  not  requisite  to  the  study  itself.  Ways  in  which  the  computer 
can  facilitate  such  experiments  include  the  following: 

Computer  automation  provides  precise  control  over  an  experiment. 

The  experimenter  may  wish  to  presents  stimulus  for  exactly  six  seconds; 
he  may  wish  to  present  information  to  the  subject  only  after  certain 
criteria  have  been  met;  he  may  wish  to  pause  when  the  subject  has 
reached  a  certain  point  in  the  experiment.  Automation  guarantees  that 
such  procedures  are  followed  exactly. 

Automation  provides  detailed  recording  of  events  as  specified  by 
the  experimenter.  Every  stimulus  given  to  the  subject  and  every  response 
made  by  the  subject  can  be  recorded,  including  time  of  occurrence.  The 
subject  need  not  be  aware  that  he  is  being  timed;  timing  by  stop  watch, 
for  example,  can  be  a  disturbing  influence  on  the  subject.  The  experi¬ 
menter  need  not  be  physically  present  in  cases  where  his  presence  might 
disturb  the  subject. 

An  automated  experiment  tends  to  be  more  uniform  in  quality.  The 
biases  that  may  arise  when  a  particular  person  monitors  the  performance 
of  a  group  of  subjects  are  minimized. 

Instantaneous  feedback  can  be  provided  the  subject.  Response- 
dependent  experiments  are  very  difficult  to  perform  manually  if  the  con¬ 
dition  for  feedback  depends  on  the  responses  in  a  complicated  manner. 
Randomized  stimuli  that  are  not  possible  with  manual  techniques  can  be 
provided  when  automation  is  used. 


Commercial  designations  are  given  only  in  the  interest  of  precise  de 
scription;  their  mention  does  not  constitute  indorsement  by  BESRL  or 
by  the  Army. 


REVIEW  OF  BESRL'S  SURVEILLANCE  RESEARCH 


The  research  studies  completed  or  in  progress  are  grouped  in  three 
categories:  1)  those  concerned  primarily  with  specific  tasks  and  func¬ 
tions  in  and  of  themselves  without  j.mnediate  regard  for  systems  use, 

2)  those  concerned  with  display  requirements  for  accomplishing  actual  or 
conceptualized  tasks,  and  5)  those  requiring  a  systems  context  for  execu¬ 
tion  because  of  concern  with  the  totality  of  system  performance  as  well 
as  with  the  separate  and  interactive  contributions  of  the  various  sub¬ 
systems  and  elements. 


Research  to  Improve  Interpretation  Techniques 

BESRL's  work  in  surveillance  research  covers  imagery  from  exotic 
sensors  such  as  infrared  and  radar,  as  well  as  photographic  imagery. 
However,  while  there  have  been  significant  advances  in  the  development 
of  exotic  imagery,  conventional  photography  still  constitutes  an  impor¬ 
tant  source  of  military  intelligence,  and  a  substantial  portion  of  BESRL's 
research  is  concerned  with  conventional  photo  interpretation.  Many  of 
the  findings  undoubtedly  carry  over  to  the  interpretation  of  other  imagery 
as  well. 

Findings  from  studies  on  interpreter  techniques  are  intended  pri¬ 
marily  for  users  of  the  system — the  interpreter,  the  G2  Air,  the  commander. 
These  findings  obviously  have  implications  for  training. 

Interpretation  of  Photographic  Imagery .  A  number  of  aspects  of 
interpretation  have  been  subjected  to  investigation,  notably  those  con¬ 
cerned  with  trade-offs  among  accuracy,  completeness,  and  time,  control  of 
imagery  viewing  procedures,  methods  for  increasing  the  proficiency  level 
of  interpreters,  and  the  interpretation  of  infrared  and  radar  imagery. 

It  became  apparent  early  in  the  research  program  that  information 
accuracy,  a  measure  of  how  much  of  what  is  reported  is  correct,  and  com¬ 
pleteness,  a  measure  of  how  much  of  what  is  actually  recorded  on  the 
imagery  has  been  correctly  extracted,  behave  differently  under  many 
identical  conditions.  Figure  1  represents  the  relationship  of  accuracy 
and  completeness  to  time. 

While  there  is  an  increase  in  completeness  as  a  function  of  increase 
in  time,  there  is  a  decrease  in  cumulative  accuracy.  This  drop  in  accu¬ 
racy  may  be  due  to  a  propensity  on  the  part  of  interpreters  to  respond 
first  to  the  obvious,  easy- to- identify  objects  which  they  can  usually 
identify  correctly.  As  they  spend  more  time  on  the  image,  they  exhaust 
the  image  and  themselves,  responding  to  increasingly  more  difficult  cues, 
with  an  increasing  rate  of  error.  Every  now  and  then,  they  pick  up  in¬ 
formation  that  is  correct  and  hence  there  is  an  increase  in  completeness. 


Figure  1  represents  families  of  curves.  With  changes  in  image 
quality  or  scale,  the  point  at  which  performance  levels  off  will  change. 
For  example,  in  one  study  (1)  utilizing  large-scale  imagery  with  rela¬ 
tively  good  ground  resolution,  accuracy  was  maintained  at  a  relatively 
high  level  for  interpretation  times  varying  from  15  seconds  to  three 
minutes  per  frame.  This  accuracy  was  accompanied  by  a  continuing  in¬ 
crease  in  completeness  to  a  very  acceptable  level.  For  poorer  image 
quality,  the  expectation  would  be  that  accuracy  would  drop  more  precip¬ 
itously,  while  completeness  would  rise  and  then  level  off  at  a  lower 
level.  Although  operationally  obtained  photo  quality  and  scale  have 
'improved  considerably  in  recent  years,  problems  of  image  quality  and 
scale  will  be  present  for  some  time  to  come,  particularly  in  the  case 
of  infrared  and  radar  imagery.  There  is  no  reason  to  believe  that 
accuracy  and  completeness  functions  will  behave  any  differently  in  the 
interpretation  of  infrared  and  radar  imagery  than  in  photo  interpretation 

In  a  study  (2)  concerning  the  performance  of  interpretation  teams, 
it  was  also  found  that  accuracy  could  be  maximized  at  the  expense  of 
completeness  and  vice  versa.  The  procedure  requires  that  two  interpreters 
interpret  the  same  photo  independently.  A  report  combining  the  two  is 
then  generated.  Accuracy  can  be  maximized  by  accepting  only  those 
target  Identifications  as  correct  that  the  two  interpreters  report  in 
common.  This  procedure,  of  course,  minimizes  completeness,  since  each 
interpreter  may  have  picked  up  correct  information  the  other  did  not  and 
this  is  lost  in  the  process.  On  the  other  hand,  accepting  all  their 
responses  as  correct  minimizes  accuracy  but  maximizes  completeness.  From 
the  point  of  view  of  information  use,  this  finding  may  be  of  great  signif¬ 
icance.  Today,  interpreters  are  not  generally  given  instructions  to 
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maximize  accuracy  at  the  expense  of  completeness  or  vice  versa.  As  a 
consequence,  the  interpreters  impose  their  own  judgment,  determining 
what  the  commandrr  will  finally  receive.  Yet  there  are  conditions  under 
which  a  commander  might  gladly  sacrifice  accuracy  for  completeness,  or 
completeness  for  accuracy.  A  commander  may  be  faced  with  the  mission  of 
having  to  eliminate  a  particular  target  under  one  of  two  sets  of  con¬ 
ditions.  In  the  first,  he  may  be  limited  in  ammunition  or  he  may  be 
constrained  from  hitting  non-targets.  In  the  second,  he  may  have  an 
ample  supply  of  ammunition  and  it  may  not  be  important  whether  he  hits 
non-targets.  Obviously,  in  the  first  case,  emphasis  is  on  accuracy  and 
in  the  second  on  completeness.  Yet  many  such  decisions  are  made,  not  by 
the  commanders  who  are  responsible  for  the  decisions,  but  by  the  inter¬ 
preters,  who  are  not  in  the  best  position  to  know  which  priorities  govern 
at  a  given  time. 

Teams,  of  course,  need  not  work  independently.  Improvement  in  per¬ 
formance  can  be  achieved  by  having  them  operate  under  different  .  proce¬ 
dures.  For  example,  the  size  of  teams  can  vary;  members  can  work  coop¬ 
eratively.  In  a  study  of  team  performance,  three-man  teams  with  indi¬ 
viduals  working  independently  provided  reports  with  considerably  higher 
accuracy  than  did  the  average  individual  interpreter  while  maintaining 
the  average  interpreter's  completeness  (3)»  Fully  cooperative  three-man 
teams,  on  the  other  hand,  showed  gains  in  completeness  while  maintaining 
the  accuracy  rate  of  the  individual.  This  latter  finding  reflecting 
gains  in  completeness  held  only  for  the  interpretation  of  relatively 
difficult  photography.  To  determine  which  aspects  of  team  operations 
are  important  in  decreasing  the  amount  of  time  required  for  team  inter¬ 
pretation  while  maintaining  superiority  of  accuracy  and/or  completeness, 
the  use  of  a  checker  who  reviews  the  initial  report  provided  by  an  inter¬ 
preter  was  investigated  f4).  The  findings  are:  When  completeness  and 
timeliness  are  essential,  team  members  should  check  each  other's  work 
without  discussion,  and  decisions  made  by  the  checker  should  constitute 
the  final  product.  When  a  greater  degree  of  accuracy  is  desired,  only 
information  agreed  upon  by  the  team  members  should  be  accepted.  Maximum 
accuracy  or  completeness  requires,  of  course,  complete  independence  of 
interpretation  on  the  part  of  the  team  members,  as  already  discussed. 

Thus,  depending  on  the  particular  requirements  and  availability  of  re¬ 
sources,  various  kinds  of  team  procedures  can  be  established  to  yield 
information  content  with  the  desired  attributes. 

Screening.  Screening  is  a  process  of  selecting,  from  a  set  of  imagery 
frames  with  high  intelligence  value  for  subsequent  hot  or  more  detailed 
interpretation.  Frequently,  the  interpreter  must  race  against  time. 
Real-time  interpretation  imposes  the  need  for  rapid  extraction  of  infor¬ 
mation  for  immediate  use.  Generally,  when  a  roll  of  imagery  is  available, 
nothing  is  known  about  what  is  depicted  in  the  roll.  By  screening,  any 
highly  significant  information  near  the  end  of  the  roll  will  be  reached 
more  quickly  than  if  interpreters  were  to  go  through  normal  interpreta¬ 
tion  procedures.  Screening  also  permits  the  determination  of  the  amount 
of  information  present  in  a  roll  and  thus  is  useful  in  deciding  whether 
the  task  of  interpreting  a  particular  roll  should  be  assigned  to  a  single 
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interpreter  or  to  two  interpreters.  If  the  volume  of  information  Is 
large,  the  roll  could  be  divided  in  half,  each  interpreter  reporting  on 
one  half.  Thus,  the  information  contained  in  the  roll  would  be  avail-, 
aole  sooner. 

In  an  early  study  (5)  utilizing  imagery  at  scales  of  approximately 
1:10,000,  screening  performance  was  found  to  be  relatively  poor  for  times 
ranging  from  5  seconds  to  50  seconds  per  frame.  Average  accuracy  was 
approximately  50$>  increasing  slightly  from  5  seconds  to  50  seconds  per 
frame.  In  a  different  study  (1),  utilizing  large  scale  imagery  (1:2000 
and  1:5000),  screening  performance  proved  considerably  better.  At  the 
fastest  rate  of  screening  (15  seconds  per  frame),  89$  of  the  frames  were 
correctly  classified. 

Another  study  (6),  also  using  relatively  large-scale  imagery  (1:1500 
to  1:5000,  yielded  a  screening  acuracy  of  79$  at  a  screening  rate  of 
approximately  10  seconds  per  frame.  With  practice,  screening  dropped  to 
approximately  8  seconds  per  frame,  some  interpreters  screening  at  rates 
of  approximately  5  seconds  per  frame.  This  result  was  substantiated  by 
a  study  (7)  also  utilizing  relatively  large-scale  imagery  (1:2000  to 
1:4000).  Screening  accuracy  improved  and  leveled  off  at  75$  at  a  screen¬ 
ing  rate  of  4  seconds  per  frame  when  rates  were  varied  from  .8  second 
per  frame  to  6  seconds  per  frame.  Thus,  it  begins  to  appear  that  screen¬ 
ing  rates  of  4  seconds  per  frame  may  be  acequate  for  relatively  large 
scale,  good  quality  photography. 

In  the  case  of  screening  followed  by  hot  reporting  (8),  it  was  found 
that  screening  adds  considerably  to  the  total  processing  time  of  the 
imagery  but  does  not  perceptibly  affect  the  accuracy  or  completeness  of 
the  interpretation.  On  this  basis,  screening  cannot  be  recommended  as  a 
time-saving  device  for  hot  reporting.  However,  this  conclusion  does  not 
rule  out  screening  as  a  technique  for  establishing  interpretation  prior¬ 
ities  or  procedures  regarding  the  allocation  of  work  to  more  than  one 
interpreter. 

Furthermore,  interpreters  looking  exclusively  for  priority  targets 
were  more  complete  and  less  accurate  than  those  looking  for  priority 
targets  in  addition  to  other  targets  on  large-scale  photography  (1:5000 
or  larger)  and  at  an  average  of  50  seconds  per  frame  (9)*  It  was  also 
found  that  the  interpreter  who  sets  his  own  pace  in  interpreting  the 
photography  is  more  accurate  than  his  mechanically-paced  counterpart. 

Real-Time  Interpretation.  Screening  and  hot  reporting  are,  of 
course,  not  real-time  interpretation.  In  real-time  interpretation,  the 
interpreter  is  generally  confronted  with  a  moving  window  display  and  he 
must  report  out  immediately  the  target  he  detects  and  identifies.  Re¬ 
porting  out  implies  not  only  naming  the  target  but  also  providing  map 
coordinates.  For  the  interpreter  to  accomplish  this  task,  the  display 
must  possess  minimum  characteristics  in  terms  of  scale,  quality,  and 
rate  of  movement.  It  must  be  linked  to  a  quick  reporting  device  and  an 
automatic  coordinate  read-out  device,  both  easily  actuated  by  the  inter¬ 
preter.  These  in  turn  must  be  linked  to  a  communication  device  or  a 
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variation  thereof  which  may  involve  voice  communication.  BESRL's  real¬ 
time  interpretation  research  is  just  getting  under  way,  but  results  from 
the  screening  studies  indicate  that  4  seconds  per  frame  are  just  about 
minimal  and  allow  hardly  sufficient  time  for  positioning  a  device,  pro-  • 
viding  the  coordinate  read-out,  and  communication.  Applicability  of  the 
results  of  the  screening  studies  rests  on  the  assumption  that  the  inter¬ 
preters  in  the  study  either  saw  targets  or  evidence  of  targets  (e.g., 
tracks)  before  flagging  a  frame  for  subsequent  interpretation.  The 
problem  of  real-time  interpretation  is  complex  and  requires  considerable 
study.  Among  the  aspects  of  concern  are  those  dealing  with  information 
loss  as  a  function  of  reporting  out  a  target.  Lucrative  targets  may  be 
missed  because  the  interpreter  can  attend  to  only  one  target  at  a  time. 

The  nature  of  the  interpretation  task  is  such  that  the  information 
output  will  be  affected  by  the  way  the  information  requirement  is  stated. 
Early,  it  was  found  that  suggestibility  must  be  considered  in  the  in¬ 
structions  to  the  interpreter  (10).  When  interpreters  were  asked  to 
"verify  the  presence  of  tanks"  in  an  image,  they  reported  numerous  tanks 
where  in  fact  none  existed.  Interpreters  asked  merely  to  report  out  the 
information  recorded  in  the  image  reported  far  fewer  tanks.  Extreme  care 
is  therefore  indicated  in  specifying  the  task  the  interpreter  is  to  per¬ 
form  to  avoid  any  suggestion  that  may  lead  to  error--or  the  verification 
of  erroneous  information. 

As  stated  previously,  trade-offs  exist  among  the  accuracy,  complete¬ 
ness,  and  speed  of  information  generated  by  interpreters.  Interpreters 
working  under  payoff  instructions  emphasizing  accuracy  and  those  in¬ 
structed  to  achieve  a  balance  between  accuracy  and  completeness  produced 
significantly  fewer  wrong  target  identifications  than  did  interpreters 
working  under  instructions  emphasizing  completeness  (11).  In  the  screen¬ 
ing  task,  those  working  under  instructions  which  emphasized  accuracy  pro¬ 
duced  significantly  fewer  wrong  detections  of  areas  of  military  activity. 
Those  with  instructions  emphasizing  completeness  produced  significantly 
more  correct  area  detections.  Another  study  (12)  compared  three  methods 
of  stating  requirements: 

1.  A  payoff  method  which  assigns  numerical  weights  to  accuracy 
versus  completeness. 

2.  A  certitude  method  which  provides  a  percent  accuracy  score  which 
must  be  met  for  information  to  be  reported. 

3«  A  verbal  method  which  states  what  is  to  be  emphasized,  accuracy 
or  completeness. 

Of  the  three  methods,  the  verbal  one  appeared  to  be  the  least  effective; 
both  other  methods  showed  promise.  Thus,  interpreters  can  vary  their 
output  as  a  function  of  the  relative  weight  given  accuracy  and  complete¬ 
ness  in  the  instructions.  However,  unless  the  interpreters  are  given 
guidance,  they  will  utilize  their  own  subjective  evaluations  of  the  in¬ 
telligence  requirements.  To  obtain  optimal  output  from  the  interpreter, 
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the  requirements  for  the  information  must  be  stated  in  the  appropriate 
manner.  A  standard  format  would  be  desirable.  With  the  objective  of 
stating  information  requirements  in  a  particular  military  situation 
quantitatively,  an  analysis  was  performed  to  develop  reliable  estimates 
of  the  respective  values  and  costs  of  correct  and  erroneous  surveillance 
information  (13).  Fifty  tactical  situations  varying  considerably  one 
from  another  were  used.  Information  requirements  in  terms  of  content 
and  cost  of  making  errors  were  empirically  obtained  from  officers  in 
command  positions.  Generally,  errors  of  omission  were  considered  most 
serious  and  misidentifications  within  a  category  of  targets  least  seri¬ 
ous.  False  alarms  and  misidentifications  across  target  categories  were 
of  intermediate  cost.  Errors  involving  certain  targets,  for  example, 
tanks  and  missiles,  were  consistently  judged  to  be  more  costly  than 
errors  on  other  targets.  While  the  procedure  was  time-consuming,  the 
feasibility  of  developing  cost  estimates  of  image  interpretation  errors 
was  demonstrated.  Such  error  cost  data  could  be  used  to  specify  imagery 
"take"  conditions  more  rigorously  and  improve  the  quality  of  the  inter¬ 
preter's  report  by  making  him  explicitly  aware  of  what  is  wanted. 

Interpreter  reports  often  are  only  partially  accurate  and  seldom 
convey  all  the  potential  information  from  the  imagery.  The  usefulness 
of  such  reports  would  be  increased  if  each  report  carried  a  valid  esti¬ 
mate  of  its  accuracy.  Normally,  interpreters  provide  estimates  of  the 
accuracy  of  their  reports  using  the  terms  "possible,"  "probable,"  and 
"positive."  These  are,  however,  subjective  judgments  and,  although 
positively  related  to  actual  performance,  far  from  perfect.  Using  sub¬ 
jective  judgments  and  objective  characteristics  of  performance,  mathe¬ 
matical  equations  were  developed  to  state  the  probability  that  informa¬ 
tion  produced  by  an  interpreter  is  correct  (14).  Predictions  generated 
from  the  selected  variables  had  substantial  validity  for  accuracy  of 
performance.  Such  predictions  could  be  used  in  place  of  confidence 
ratings  or  could  be  used  to  weight  the  identifications  made  by  several 
interpreters. 

Interpretation  of  Infrared  and  Radar  Imagery.  In  the  area  of  radar 
(SLAR)  and  infrared  (IR)  interpretation,  considerable  work  has  been  done 
on  the  interpretability  of  displays  generated  by  these  sensors.  In  the 
case  of  SLAR,  research  concerned  the  interpretability  of  images  in  two 
modes,  the  normal  fixed  target  indication  mode  and  the  moving  target 
indication  mode.  In  addition,  a  SLAR  change  detection  study  was  accom¬ 
plished.  In  the  case  of  infrared  imagery,  interpretability  data  were 
obtained  for  various  targets  as  a  function  of  varying  acquisition  con¬ 
ditions.  In  addition,  a  study  was  conducted  concerning  the  effects  of 
trading  thermal  sensitivity  for  spatial  resolution  or  vice  versa.  Work 
is  currently  in  progress  for  the  development  of  an  infrared  reference 
key  for  interpretation. 


Research  on  Displays 


Research  findings  from  studies  on  displays  provide  information  on 
display  requirements  based  on  interpreter  needs.  These  findings  are  of 
more  iirmediate  use  to  those  concerned  with  the  development  of  interpreta¬ 
tion  equipment  and  systems--system  designers  and  engineers,  in  particular. 
The  nature  of  the  problems  implies  the  need  for  discrete  studies,  since 
display  requirements  are  of  necessity  tied  to  specific  requirements 
imposed  on  interpreters. 

One  of  the  earlier  studies  in  this  area  was  the  evaluation  of 
stereoscopic  viewing  for  photo  interpretation  (15).  Stereo  interpreta¬ 
tion  requires  use  of  overlapping  imagery  which  in  turn  has  implications 
for  imagery  procurement  and  cost.  Results  indicate  that,  for  general 
purposes,  stereo  viewing  does  not  provide  better  performance  in  either 
accuracy  or  completeness  than  non-stereo  interpretation. 

Stereo  imagery  imposes  a  need  for  a  60>  imagery  overlap,  with  por¬ 
tions  of  the  same  area  depicted  in  successive  image  frames.  Studies  were 
conducted  to  determine  the  effect  of  this  overlap  on  performance  under 
conditions  when  interpreters  do  not  use  stereo- -apparently  most  of  the 
time.  Findings  indicate  that  with  overlapping  imagery  there  was  no  in¬ 
crease  over  the  no-overlap  condition  in  accuracy  of  screening,  negligible 
increases  in  completeness,  and  a  substantial  increase  in  interpretation 
time  (6).  A  similar  finding  was  obtained  for  the  actual  interpretation 
of  overlapping  imagery  (8).  Based  on  these  findings,  it  becomes  evident 
that  the  procurement  and  display  of  overlapping  imagery  may  be  detrimental 
under  many  conditions.  The  elimination  of  imagery  overlap  will  reduce 
considerably  the  time  required  for  the  preparation  of  intelligence  re¬ 
ports.  In  addition,  the  number  of  interpreters  required  to  handle  a 
given  workload  can  be  reduced;  or  a  larger  number  of  missions  can  be 
interpreted  within  a  given  period  of  time  and  with  the  same  resources. 

The  load  imposed  on  the  facility  providing  photo  processing  support  will 
also  be  reduced  substantially.  Consequently,  it  would  appear  that 
imagery  should  generally  not  be  procured  with  the  overlap  required  for 
stereo  viewing.  If  stereo  is  desired  for  purposes  such  as  height  deter¬ 
mination,  it  should  be  requested  for  this  purpose  on  an  a  priori  basis. 

Another  study  compared  performance  in  the  interpretation  of  positive 
and  negative  images  (16).  Concern  here  was  with  the  need  for  presenting 
interpreters  with  positive  images  which  require  an  additional  processing 
step.  Findings  indicate  no  differences  in  performance  in  the  interpreta¬ 
tion  of  positive  and  negative  transparencies.  However,  the  subjects  were 
experienced  interpreters.  Some  training  in  the  interpretation  of  nega¬ 
tives  might  be  required  to  bring  interpreter  trainee  performance  up  to 
that  attained  with  positives.  In  deciding  which  to  use,  the  cost  of 
training  must  be  balanced  against  the  cost  of  providing  positives. 

Considering  image  interpretation  systems  for  field  use,  many  ques¬ 
tions  arise  concerning  alternative  viewing  devices.  One  choice  to  be 
made  is  between  light  tables  and  rear  projection  systems.  Both  devices 


permit  the  display  of  roll  film  for  interpretation.  There  are,  of  course, 
major  differences.  Everything  else  being  equal,  mobility  requirements 
imposing  size  and  weight  constraints  would  clearly  point  to  the  light 
table  as  preferable.  But  everything  else  is  not  equal,  and  many  studies 
need  to  be  conducted  and  possible  trade-offs  considered  in  arriving  at 
an  intelligent  decision.  In  an  exploratory  study,  keeping  levels  of 
magnification  equal  for  both,  it  was  found  that,  in  detailed  interpreta¬ 
tion  and  screening,  there  are  no  significant  differences  in  performance 
between  light  table  and  rear  projection  system  (17)*  For  hot  reporting, 
however,  the  light  table  proved  superior.  These  findings,  however,  can¬ 
not  be  considered  conclusive  at  this  time.  Moreover,  in  addition  to  the 
general  interpreter  functions — screening,  hot  reporting,  and  detailed 
reporting — interpreters  also  compare  late  cover  with  earlier  cover  for 
the  purpose  of  change  detection.  Comparative  cover  interpretation  in¬ 
volves  the  use  of  two  sets  of  imagery  covering  the  same  ground  area,  one 
obtained  initially  and  one  flown  later.  The  interpreter  compares  two 
images  of  the  same  area  to  determine  what,  if  any,  change  has  occurred 
in  the  interval  between  the  two  missions.  Operationally  flown  compara¬ 
tive  cover  imagery  is  rarely  congruent  and  there  are  frequently  differ¬ 
ences  in  orientation  and  scale  of  imagery  obtained  in  the  two  missions. 
Departure  from  congruence  results  in  degradation  in  both  accuracy  and 
completeness  (18).  A  similar  finding  was  also  obtained  for  just  locating 
the  appropriate  image  on  the  later  cover  (19).  This  result  would  argue 
for  providing  the  capability  of  rotating  and  variably  magnifying  the  two 
images  so  that  they  can  be  brought  into  congruence.  At  this  time,  light 
tables  do^pot  provide  such  capabilities.  However,  rear  projection 
systems  do. 

Comparative  cover  interpretation  requires  the  retention  of  older 
cover  for  use  when  more  recent  cover  is  obtained,  imposing  a  need  for 
storage  space  which  is  at  a  premium  in  mobile  Army  systems.  If  change 
detection  can  be  accomplished  by  means  other  than  comparative  cover 
interpretation,  then  the  need  for  such  space  could  be  voided.  With  com¬ 
puterized  interpretation  facilities,  it  may  be  possible  for  the  imagery 
to  be  interpreted  as  it  comes  in  and  the  intelligence  information  stored 
in  the  computer.  As  new  imagery  for  the  same  area  is  interpreted,  the 
information  can  also  be  input.  Computer  matching  of  the  two  reports,  by 
map  coordinates,  can  be  used  to  generate  change  reports.  Iii  a'BESRL 
study  of  change  detection  methods,  performance  using  the  computer  method 
was  never  exceeded  by  any  of  the  comparative  cover  interpretation 
methods  (20).  However,  prior  to  concluding  that  the  computer  method  is 
an  acceptable  method,  certain  verifying  experiments  need  to  be  conducted. 

In  addition  to  the  presentation  of  imagery,  projection  systems  can 
be  used  to  display  maps  to  obtain  map  coordinates.  The  display  of  map 
information  via  projection  has  the  advantage  that  materials  may  be  stored 
on  slides  or  chips.  Projection  would  also  permit  a  more  appropriate 
interface  with  a  computer  than  is  possible  with  hard  copy  material.  The 
display  of  map  information  on  a  screen  rather  than  on  hard  copy  increases 
the  time  taken  to  locate  an  image  on  a  map  and  to  determine  the  map  coor¬ 
dinates  of  an  object  on  the  map  (21).  The  increased  time  to  use  a  screen 
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was,  however,  primarily  due  to  the  fact  that  an  imaged  area  frequently 
falls  at  the  boundary  of  two  map  chips.  Coordinate  determination  was 
not  affected  by  the  nature  of  the  display.  More  definitive  studies  need 
to  be  conducted  to  determine  if  the  time  for  image  location  on  projected 
maps  can  be  cut . 

The  Armed  Forces  are  in  process  of  instituting  a  procedure  by  which 
map  coordinate  location  data  and  other  such  platform  and  sensor  data  are 
recorded  in  a  code  matrix  data  block,  in  a  corner  of  each  individual 
image  frame.  The  coordinate  location  and  other  data  are  in  binary  coded 
form,  requiring  either  a  program  to  train  interpreters  to  read  the  block, 
or  a  decoding  device  that  would  transform  the  information  into  alpha¬ 
numeric  form.  It  was  found  that  with  a  training  program  requiring  little 
more  than  one  hour,  interpreters  could  be  taught  to  read  this  coded  in¬ 
formation  rapidly  with  an  accuracy  of  Q&p  requiring  little  expenditure 
in  time  (22).  It  would  t-herefore  appear  that  alpha-numeric  display  of 
this  information  may  not  be  necessary.  However,  in  going  into  computer¬ 
ized  systems  where  there  may  be  a  requirement  to  automatically  read  the 
information  in  the  block  for  entry  into  the  computer,  it  may  be  possible 
to  provide  this  alpha-numeric  display  for  relatively  little  added  cost. 
Consequently,  the  problem  whether  to  provide  such  an  alpha-numeric  dis¬ 
play  may  be  a  moot  question.  On  the  other  hand,  the  training  of  inter¬ 
preters  to  read  the  binary-coded  matrix  data  block  would  still  appear  to 
be  desirable,  considering  the  need  for  manual  backup  to  the  computerized 
systems- -which  do  fail  now  and  then. 

The  interpretability  of  vertical  and  oblique  photographs  was  the 
subject  of  another  st  •  (23).  Objects  with  major  dimensions  in  the 
horizontal  plane  were  better  detected  and  identified  on  vertical  views; 
objects  with  major  dimensions  in  the  vertical  plane  were  better  detected 
and  identified  on  the  oblique  views.  It  was  also  found  that  objects  in 
the  foreground  of  oblique  views  were  generally  better  detected  than  those 
in  the  foreground  on  the  vertical  views.  The  reverse  was  found  for 
objects  in  the  background  on  the  oblique  view.  Considering  that  for 
forward  oblique  imagery  what  is  background  on  a  given  frame  may  be  fore¬ 
ground  on  subsequent  frames,  it  may  be  desirable  for  some  purposes  to 
acquire  oblique  imagery.  For  mensuration  purposes,  however,  the  oblique 
view  is  inferior  to  the  vertical  vietf.  Also  to  be  considered  is  that 
these  findings  come  from  a  study  where  the  average  scale  on  the  oblique 
view  was  essentially  that  of  the  vertical  view. 

Recent  review  and  study  of  the  Army's  photo  interpretation  kit, 
which  includes  the  most  common  tools  of  the  interpretation  trade  such  as 
magnifiers,  stereo  viewers,  and  measuring  scales,  led  to  the  conclusion 
that  all  the  items  in  the  PI  kit  may  be  used  at  one  time  or  another. 

Not  all  the  tools  needed  for  a  complete  capability  are  included  in  the 
kits  (24).  With  emphasis  on  the  need  for  additional  aids  for  mensuration 
and  plotting,  it  was  further  concluded  that  an  abbreviated  kit  should  be 
made  available  for  personal  use  for  activities  performed  away  from  an 
interpretation  facility.  The  remainder  of  the  items  should  be  issued 
from  the  facility's  supplies  for  use  by  interpreters  within  the  facility. 
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As  part  of  this  effort,  a  new  design  for  tube  magnifier  reticles  was  con¬ 
ceptualized,  as  were  also  plotting  templates  for  panoramic  Imagery — 
currently  not  'available.  Steps  should  be  taken  to  provide  interpreters 
with  these  modified  magnifiers  and  plotting  templates. 

Interpretation  Keys  and  References .  Image  interpreters  often  make 
use  of  interpretation  keys  or  references.  These  keys  depict  objects  and 
can  thus  be  used  as  comparators  in  making  a  particular  interpretation. 

For  example,  if  an  interpreter  suspects  that  a  particular  object  in  an 
image  is  a  specific  kind  of  tank,  he  can  go  to  the  key  and  compare  the 
imaged  object  with  a  picture  of  the  tank.  Keys  can  take  different  forms; 
for  example,  the  key  just  described  is  a  "rights"  key  depicting  objects 
as  they  appear.  Another  type  of  key  is  an  "error"  key,  a  reference  that 
alerts  interpreters  to  errors  commonly  made  in  interpretation.  Error 
keys  proved  superior  to  rights  keys  in  affecting  performance  ( 2 5). 
Essentially,  the  error  keys  improved  accuracy  without  adversely  affecting 
completeness . 

As  part  of  BESRL's  effort,  a  number  of  experimental  keys  have  been 
developed  including  a  mini-key  for  U.  S.  equipment  which  can  be  used  as 
a  handy  condensed  reference  ( in  place  of  the  large  and  bulky  keys  cur¬ 
rently  available)  and  a  Vietnam  error  key  which  considerably  improves 
interpreter  performance.  In  addition  to,  or  instead  of,  photographic 
Vsys,  schematic  presentations  can  be  used.  Different  views  of  an  object 
can  be  presented  and  at  different  photo  scales.  It  appears  (26)  that 
schematic  presentations  are  as  good  as  photographic  presentations.  So 
far,  BESRL's  work  on  keys  has  concerned  itself  primarily  with  photo¬ 
graphic  imagery.  More  recently,  work  has  been  initiated  to  determine 
what  types  of  key  are  best  for  use  with  infrared  imagery. 

Research  on  Image  Quality.  The  quality  of  the  image  is  of  major 
concern  to  image  interpretation.  From  a  general  point  of  view,  there  is 
a  direct  relationship  between  image  quality  and  the  interpretability  of 
an  image.  An  understanding  of  the  relationship  between  performance  and 
image  quality  would  permit  determination  of  the  points  of  diminishing 
returns  for  image  quality.  Moreover,  estimates  of  the  quality  of  imagery 
obtained  in  the  field  could  be  very  useful  for  determining  the  inter¬ 
pretability  of  the  imagery  for  particular  purposes  and  could  be  used  to 
reduce  the  amount  of  imagery  to  be  interpreted.  Furthermore,  measure¬ 
ment  of  image  quality  in  the  field  can  be  an  aid  in  determining  the 
probability  of  the  accuracy  of  a  particular  interpretation. 

However,  image  quality  is  not  a  unitary  dimension.  Factors  such  as 
granularity,  acutance,  contrast,  resolution,  and  scale  are  all  components 
of  image  quality.  Definitive  studies  relating  these  dimensions  to  per¬ 
formance  would  be  of  very  large  magnitude.  Experimentation  focusing  on 
factors  that  may  have  an  effect  on  image  quality  and  that  can  be  con- 
trolled--or  compensated  for--was  considered  more  feasible.  Research  was 
therefore  conducted  on  factors  of  image  motion,  atmospheric  haze,  and 
scale  as  they  affect  image  degradation  (27).  Results  indicate  that  these 
factors  can  be  accounted  for  by  a  single  factor,  the  ground  resolvable 
distance . 
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Since  measurement  of  image  quality  could  be  exceedingly  useful  in 
the  field,  a  study  of  the  effects  of  quality  judgments  by  interpreters 
regarding  the  interpretability  of  images  was  undertaken  (28).  The 
average  judgments  by  a  number  of  interpreters  proved  very  reliable  and 
possessed  a  fair  amount  of  validity.  An  image  quality  catalog,  was  de¬ 
veloped  for  use  by  interpreters.  This  catalog  presents  images  in  terms 
of  three  dimensions:  scene  complexity,  photo  scale  and  image  sharpness. 
Data  on  the  interpretability  of  the  images  were  obtained  and  the  feasi¬ 
bility  of  using  these  images  to  obtain  measurements  of  image  quality  was 
demonstrated  (29).  The  catalog  measures  were  found  to  be  considerably 
more  valid  than  average  quality  judgments  made  by  interpreters.  The 
interpreter  using  this  catalog  compares  the  images  he  is  interpreting 
with  the  images  in  the  catalog  and  selects  the  image  in  the  catalog  that 
most  closely  resembles  in  quality  the  image  he  is  interpreting.  This 
preliminary  study  merely  established  the  feasibility  of  the  approach. 

The  general izabil ity  of  the  catalog  requires  further  experimentation. 

In  recent  years,  the  transmission  of  imagery  from  aircraft  to  ground 
has  come  to  be  of  major  concern.  Real-time  or  near  real-time  interpreta¬ 
tion  is  becoming  increasingly  more  important  and  hence  the  time  saved  by 
not  having  to  wait  for  the  return  of  the  aircraft  becomes  significant. 
Further,  with  real-time  transmission,  the  receipt  of  the  imagery  may  make 
a  mission  at  least  partly  successful  even  if  the  aircraft  does  not  return. 
Unfortunately,  the  higher  the  image  resolution  desired,  the  more  band¬ 
width  is  required  for  transmission,  and  large  amounts  of  imagery  to  be 
transmitted  at  high  resolution  could  preempt  the  bandwidth  required  for 
other  messages.  Although  transmission  time  can  be  traded  for  bandwidth 
to  achieve  a  higher  level  of  resolution,  this  step  does  not  solve  the 
problem,  because  of  the  ever-increasing  time  delay  from  transmission  to 
receipt  of  the  imagery. 

From  the  point  of  view  of  the  utility  of  the  imagery  received  on 
the  ground,  the  problem  reduces  to  determining  the  image  scale-quality 
required  for  the  interpreter  to  be  able  to  interpret  the  imagery,  or  at 
least  to  determine  which  portion  of  the  imagery  should  be  retransmitted 
with  more  resolution.  The  latter  problem  concerning  screening  was  in¬ 
vestigated  in  a.i  exploratory  study  (7).  The  interpreter's  ability  to 
correctly  classify  70mm  photo  frames  at  a  scale  varying  from  1:2000  to 
1:4000  and  at  screening  rates  varying  from  .8  second  per  frame  to  6 
seconds  per  frame  declines  as  scale,  image  quality,  and  time  allotted 
for  the  task  decreases.  Maximizing  any  of  these  three  dimensions  re¬ 
sults  in  improved  performance.  Thus,  poorer  quality  imagery  can  be 
adequately  screened  by  allowing  more  time  for  screening.  More  definitive 
studies  are  planned  to  home  in  on  the  resolution,  scale,  and  screening 
rate  requirements  for  adequate  screening  of  transmitted  imagery. 
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SYSTEMS  STUDIES 


BESRL's  systems  studies  are  concerned  with  a  number  of  different 
facets  relating  to  system  development  and  system  use.  They  serve  the  pur¬ 
poses  of  integrating  findings  from  many  relevant  efforts,  of  conceptualiz¬ 
ing  and  evaluating  system  alternatives,  and  of  conceptualizing  and  testing 
various  aspects  of  system  and  computer  use  to  maximize  the  capability  of 
the  system.  The  expectation  underlying  much  of  this  effort  is  that  future 
interpretation  facilities  will  have  computers,  and  that  these  computers 
can  serve  for  many  more  purposes  than  the  mere  storage  and  letrieval  of 
data  and  the  computation  required  for  obtaining  map  coordinates.  Within 
this  area,  BESRL  is  conducting  research  to  develop  and  use  a  systems  mea¬ 
surement  methodology,  developing  procedures  by  means  of  which  the  computer 
can  assist  m<m  in  the  decision  process,  and  determining  how  best  to  use 
computerized  interpretation  facilities  for  the  maintenance  or  improvement 
of  interpreter  proficiency. 

Systems  Measurement  Methodology 

The  Information  Systems  Laboratory  provides  the  physical  facility 
necessary  for  total  system  research  in  the  area  of  image  interpretation. 
Within  this  laboratory,  a  wide  variety  of  image  interpretation  systems 
can  be  configured  for  simulation  purposes  even  though  the  system  to  be 
simulated  is  only  in  the  conceptualization  (pre-QMR)  stage.  However, 
meaningful  systems  reseazch  in  terms  of  performance  evaluation  requires 
a  vehicle  or  methodology  for  exercising  the  simulation.  Such  a  method¬ 
ology  should  permit  realistic  simulation  of  image  interpreter  facility 
operations  in  a  tactical  situation  with  sufficient  laboratory  control 
for  derivation  of  pertinent  quantitative  performance  measures.  To  that 
end,  a  Standard  Systems  Measurement  Bed  (SSMB)  was  developed,  and  a  semi- 
automated  Tactical  Image  Interpretation  Facility  (TIIF)  was  conceptualized 
and  simulated  as  a  means  of  evaluating  the  utility  of  the  SSMB  in  com¬ 
paring  image  interpretation  systems  at  the  system,  subsystem,  function, 
and  subfunction  levels  (30).  Constructing  the  SSMB  involved  three  main 
tasks:  1)  identifying  typical  image  interpretation  processes  to  be  rep¬ 
resented,  2)  review  of  available  materials  and  selection  of  those  pro¬ 
viding  suitable  problems  and  exercises  in  the  context  of  a  reasonable 
scenario,  and  3)  determination  of  appropriate  measures  and  data  collec¬ 
tion  techniques  to  insure  adequacy  and  interpretability  of  results.  The 
ramifications  of  these  steps  are  described  in  Appendix  B. 

Corps  level  was  selected  as  the  echelon  of  operation  to  be  simulated 
because  it  has  organically  assigned  aircraft  and  would  have  the  diversity 
of  activities  and  requirements  to  impose  a  taxing  workload  on  a  TIIF. 
Guided  by  Army  doctrine  for  corps  level  operations,  the  desired  test 
characteristics  were  specified  for  the  major  test  components:  inputs, 
requirements,  and  outputs  as  shown  in  Table  B-2.  On  the  above  basis, 
imagery  was  selected,  SOP's  and  reporting  requirements  were  established, 
and  supporting  documentation  ( as  detailed  in  Appendix  B)  was  prepared. 

A  coherent  scenario  was  written  to  effectively  bond  together  all  the 
SSMB  elements.  Concurrently,  data  collection  and  processing  plans  were 
formulated. 
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Several  trial  runs  with  subsequent  modification  and  refinement  re¬ 
sulted  in  a  package  of  selected  and  scheduled  input  materials,  uniform 
and  realistic  processing  requirements,  and  systematic  data  collection 
and  scoring  techniques  imbedded  in  a  reasonable  5-day  scenario.  The 
package  consists  of  56  rolls  of  imagery,  including  infrared,  side¬ 
looking-radar,  and  a  variety  of  day  and  night  photography  representing 
the  quantities  and  types  that  might  be  acquired  during  an  actual  mili¬ 
tary  operation.  The  SSMB  is  sufficiently  open-ended  that  new  types  of 
imagery  or  terrain  and  cover  can  easily  be  added  if  circumstances 
warrant.  All  materials  and  procedures  for  administration,  as  well  as 
instructional  materials  for  the  image  interpreter  subjects,  are  docu¬ 
mented.  Figure  2  shows  how  the  SSMB  would  be  implemented. 

The  final  evaluation  of  the  SSMB  involved  full-scale  application 
in  the  BESRL  Information  Systems  Laboratory.  This  application  was  a 
five- step  process:  1)  conceptualization  of  a  representative  or  model 
system  to  be  evaluated,  2)  building  a  simulation  of  the  system  in  the 
laboratory,  5)  exercising  the  simulation  using  the  SSMB,  4)  collection 
and  analysis  of  performance  data  during  the  exercise,  and  5)  evaluating 
the  performance  of  the  simulated  system  as  a  means  of  assessing  the  SSMB. 

The  experimental  model  concept  was  based  primarily  on  analysis  of 
Army  requirements  for  a  TIIF  in  the  current  time  frame.  These  require¬ 
ments  were  used  to  define  the  basic  functions  and  tasks  needed  for  a 
fieldable  semiautomated  TIIF.  Steps  involved  in  the  model  development 
are  described  in  Appendix  B.  The  model  TIIF  specifies  six  image  inter¬ 
pretation  stations  which  can  accommodate  up  to  eight  interpreters.  The 
floor  plan  of  the  model  is  shown  in  Figure  5*  Guidelines  were  estab¬ 
lished  for  construction  of  the  simulation  in  the  laboratory  with  a  goal 
of  duplicating  the  model  in  all  functional  aspects  and  closely  approxi¬ 
mating  it  in  terms  of  physical  dimensions  and  configuration  of  equipment. 
The  floor  plan  (Figure  4)  illustrates  how  closely  the  laboratory  simula¬ 
tion  resembles  the  conceptualized' model . 

A  series  of  full-scale  tests  of  the  SSMB  was  conducted,  starting 
with  a  portion  of  the  system  and  gradually  involving  more  and  more  of 
the  system  until  the  total  system  was  included.  Trained  image  inter¬ 
preters  manned  the  system  for  the  five  days  of  each  test.  Data  were 
collected  during  the  tests.  At  the  end,  the  experimenter  and  observers 
collaborated  to  reconstruct  all  test  events  and  to  score,  reduce,  and 
analyze  the  data.  Normally,  evaluation  involves  assessing  total  system 
performance  in  terms  of  established  criteria  and  ascertaining  the  rela¬ 
tive  contribution  of  the  various  subsystems,  functions,  and  subfunctions. 
Various  man-procedure-equipment  factors  are  carefully  examined  to  find 
ways  in  which  the  configuration  might  be  changed  to  provide  improved  per¬ 
formance  on  subsequent  iterations.  In  the  present  instance,  the  purpose 
was  to  evaluate  the  SSMB  as  a  vehicle  for  obtaining  the  required  data, 
and  the  analyses  focused  on  the  pertinence  and  interpretability  of  the 
data. 
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Figure  2.  Implementation  of  the  SSMB  in  the  BESRL  Information  Systems  Laboratory 
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♦Stations  A  through  E  have  access  to  the  computer.  Scale  1"  -  3  feet 

Figure  3.  Tactical  Image  Interpretation  Facility  model  concept 


Figure  4.  Floor  plan  for  the  laboratory  simulation 
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While  not  yet  completed ,  the  analyses  to  date  clearly  indicate  the 
feasibility  of  the  SSMB  for  its  stated  purpose.  Twelve  different  time¬ 
liness  measures,  13  accuracy  measures,  and  12  completeness  measures  were 
obtained.  Additional  scores  could  be  derived  by  combining  and  or  weight¬ 
ing  the  basic  measure!-  in  a  variety  of  ways  to  reflect  the  relative 
importance  the  intelligence  staff  assigns  to  accuracy,  completeness,  and 
timeliness,  as  well  as  to  type  and  content  of  the  image  interpreter  re¬ 
ports.  Diagnostic  measures  were  also  obtained  on  the  performance  of  each 
man-station  combination,  the  relative  amount  of  time  spent  in  each  of  the 
various  functions  and  subfunctions,  and  the  various  verbal,  transfer,  and 
movement  links  of  man  and  station. 

The  SSMB  has  been  demonstrated  to  be  a  valuable  tool  for  assessing 
alternative  image  interpretation  systems  and  system  configurations  while 
the  systems  are  still  in  the  conceptualization  (pre-QMR)  stage  or  early 
development  (before  prototype)  stage.  It  permits  evaluation  of  total 
system  effectiveness  and  of  the  contribution  of  the  various  system  ele¬ 
ments  to  that  effectiveness.  It  enables  dissection  of  man- procedure- 
equipment  interactions,  providing  clear  indications  of  how  such  relation¬ 
ships  can  be  optimized  for  improved  efficiency  and  effectiveness.  The 
SSMB,  together  with  the  Information  Systems  Laboratory,  provides  a  unique 
capability  for  realizing  appreciable  savings  in  time,  effort,  and  funds 
through  empirical  determination  of  system  design  and  operating  character¬ 
istics  early  in  the  research  and  development  cycle  rather  than  through 
the  more  customary  start- stop-redesign  or  retrofit  route. 


Man/Computer  Decision  Processes 

In  a  systems  context,  image  interpreters  and  the  system  computer  can 
be  viewed  as  components  whose  task  is  to  perform  their  functions  so  as  to 
contribute  maximally  to  the  system  output.  The  system  output  is  intended 
to  satisfy  levied  mission  requirements.  The  study  of  man/computer  deci¬ 
sion  processes  is  an  attempt  to  increase  the  meaningfulness  and  utility 
of  the  interpretation  system  output  by  tailoring,  with  the  aid  of  a  com¬ 
puter,  the  quality  and  quantity  of  interpreter  reports  to  the  needs  of 
the  military  situation. 

A  series  of  research  studies  is  being  conducted  to  determine  whether 
a  general  decision  model  involving  the  evaluation  of  action  alternatives 
through  the  use  of  pay-off  matrices  and  probability  estimates  can  be  used 
to  help  control  interpreter  output.  The  pay-off  matrix  states  the  rela¬ 
tive  value  of  each  action  alternative  (in  this  case,  making  various 
alternative  interpretation  reports),  given  each  possible  true  state  or 
condition  (in  this  case,  the  actual  targets  in  the  area  covered  by  the 
photographs).  The  model  is  probabilistic  in  the  sense  that  it  admits 
that  the  true  state  or  condition  confronting  the  decision  maker  is 
seldom  positively  known,  and  it  is  necessary  to  hypothesize  a  set  of 
mutually  exclusive  and  exhaustive  true  states  and  to  estimate  their  re¬ 
lative  probability  (in  this  case,  the  probability  that  the  object  being 
interpreted  is  each  of  the  targets  for  which  the  interpreter  is  searching). 


In  the  general  model,  when  the  values  in  the  pay-off  matrix  are 
weighted  or  multiplied  by  the  probabilities  of  the  true  states,  the 
product  is  the  expected  value  of  each  of  the  decision  alternatives. 

In  the  application  to  image  interpretation,  a  simplification  can  be 
effected  through  considering  only  the  costs  of  making  erroneous  inter¬ 
pretation  reports.  Under  the  assumption  that  a  correct  interpretation 
has  no  cost,  zeros  can  be  inserted  in  the  pay-off  matrix  for  every  re¬ 
port  alternative  corresponding  exactly  with  a  true  state.  This  pro¬ 
cedure  makes  the  task  of  obtaining  values  to  place  in  the  matrix  con¬ 
siderably  easier,  as  only  costs  need  be  considered  rather  than  both 
costs  and  gains.  The  cost  matrix,  when  multiplied  by  the  target  prob¬ 
ability  vector,  yields  the  expected  cost  of  each  reporting  alternative; 
the  decision  rule  is  to  choose  the  alternative  with  the  minimum  expected 
cost. 


To  apply  the  expected  cost  approach  to  an  image  interpretation 
system,  two  types  of  data  are  needed.  The  interpreter  would  provide  the 
information  on  which  the  target  probability  vector  would  be  generated. 

The  cost  data  would  have  to  be  supplied  by  the  commander  or  his  staff  in 
terms  of  some  arbitrary  unit  denoting  the  relative  seriousness  or  cost  to 
current  mission  of  each  of  the  possible  image  interpreter  errors.  Table  1 
contains  an  abbreviated  sample  matrix  with  a  set  of  hypothetical  costs. 


Table  1 

ABBREVIATED  SAMPLE  MATRIX  SHOWING  HYPOTHETICAL  DATA  FCR 
EXPECTED  COSTS  IN  PHOTOINTERPRETATION  SYSTEMS 


•Identification 

Tank 

Truck 

TRUTH 

Missile 

Nothing 

Probability 

Vector 

Expected 

Cost 

Tank 

0 

2 

5 

4 

.40 

2.3 

Truck 

3 

0 

6 

2 

.20 

3.2 

Missile 

2 

2 

0 

5 

.30 

1.7 

Nothing 

8 

4 

14 

0 

.10 

8.2 

Vehicle 

3 

1 

6 

2 

3.4 

Weapon 

3 

2 

4 

2 

3-0 

The  highest  cost  in  the  matrix  is  the  error  of  omitting  a  missile — 
reporting  nothing  when  in  fact  a  missile  is  there.  The  lowest  cost  is 
the  imprecision  error  of  calling  a  truck  simply  a  vehicle- -some  small 
cost  is  associated  with  insufficient  specificity.  Misidentification 
errors  (identifying  a  tank  as  a  truck)  and  inventive  errors  (identifying 
a  non-target  image  as  a  target)  are  usually  intermediate  in  cost.  Note 
that  perfect  information  has  no  cost.  In  the  example  the  image  inter¬ 
preter  has  identified  a  given  object  on  the  imagery  as  most  probably  a 
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tank.  However,  there  is  only  a  40#  probability  that  the  given  object  is 
a  tank,  20#  that  it  is  a  truck,  30#  that  it  is  a  missile,  and  10#  that 
it  is  really  not  a  military  object  at  all.  When  the  cost  matrix  is 
multiplied  by  the  associated  probabilities,  the  expected  costs  of  the 
reporting  alternatives  are  produced  as  shown  in  the  last  column.  In  this 
instance,  the  lowest  expected  cost  is  for  the  response  'fissile."  There¬ 
fore,  the  object  in  question  would  be  reported  as  a  "Missile"  even  though 
it  is  more  probably  a  tank. 

One  way  such  information  can  be  used  is  to  control  the  quantity  and 
quality  of  information  output  by  the  system  as  shown  in  Figure  5.  The 
intelligence  officer  (G2)  to  whom  the  image  interpretation  facility's 
report  is  being  sent  can  set  a  quality  standard  for  acceptance  of  the 
information.  This  standard  is  a  single  number  representing  the  maximum 
acceptable  expected  cost.  Suppose,  in  the  example,  the  G2  has  set  this 
maximum  cost  at  2.0.  Only  responses  with  an  expected  cost  at  or  below 
this  maximum  cost  would  be  reported.  The  missile  response  would  qualify 
even  though  the  probability  that  the  target  is  truly  a  missile  is  only 
30#.  If  the  G2  had  set  a  more  stringent  cost,  the  system  would  be  forced 
to  improve  the  probability  of  the  identification  through  using  other 
interpreters  or  seeking  corroborative  information  or  better  photographic 
coverage.  Setting  a  gererous  standard  allows  the  system  to  produce  a 
large  amount  of  information,  but  the  average  accuracy  of  the  information 
may  be  low. 

On  a  conceptual  level,  the  expected  cost  approach  seems  to  have 
merit.  Yet  the  implementation  of  such  a  procedure  is  dependent  not  only 
on  having  a  computer  in  the  system,  but  also  on  obtaining  reliable  prob¬ 
ability  estimates  from  the  interpreters  and  valid  costs  from  the  com¬ 
mander  or  his  staff.  A  final  consideration  is  that  obtaining  these  data 
not  impose  unreasonable  time  demands  on  the  personnel  involved.  BGSRL 
has  undertaken  research  to  develop  the  appropriate  methodology  for  making 
practicable  both  the  estimation  of  probabilities  and  the  construction  of 
error  cost  matrices.  Some  of  these  studies  are  described  below. 

One  study  in  the  probability  area  (14)  investigated  the  feasibility 
of  devising  a  more  systematic  method  of  predicting  the  accuracy  of  re¬ 
ports  made  by  photo  interpreters.  Using  simple  and  easily  obtainable 
subjective  judgments  and  objective  characteristics  of  an  interpreter's 
performance,  mathematical  equations  were  developed  to  estimate  the  prob¬ 
ability  that  information  produced  by  an  interpreter  is  in  fact  correct. 
These  equations  have  substantial  validity  in  predicting  the  accuracy  of 
interpreter  performance.  However,  procedures  for  obtaining  the  equations 
are  somewhat  cumbersome  and  the  approach  is  not  a  very  efficient  way  to 
generate  a  probability  vector  in  comparison  with  the  single  probability 
prediction.  Should  changes  affecting  the  variables  occur  over  time,  the 
requirement  to  update  and  modify  the  equation  imposes  considerable  burden 
on  the  system.  Consequently,  it  was  decided  to  investigate  intensively 
the  confidence  judgment  variable,  which  in  essence  is  a  direct  probabil¬ 
ity  estimate  made  by  the  interpreter,  since  it  was  one  of  the  strongest 
predictors  in  the  above  study  and  would  seem  to  lend  itself  readily  to 
the  generation  of  an  entire  probability  vector. 


The  study  compared  three  techniques  for  obtaining  estimates  of  prob¬ 
able  accuracy:  the  Direct  Name  Vector  (!)NV),  the  Derived  Name  Vector- 
Experimental  (DNVE),  aid  the  Derived  Vector-Control  (DNVC).  The  DNV 

technique  imposes  the  f?sk  of  directly  judging  the  total  target  probabil¬ 
ity  and  the  probabilities  of  selected  specific  target  names.  The  DNVE 
technique  requires  direct  estimation  of  the  probability  that  the  object 
belongs  in  a  general  target  category,  as  well  as  the  probability  that  it 
belongs  at  three  specific  levels  of  target  description,  including  the  most 
likely  specific  target  name.  The  DNVC  technique  imposes  the  task  of 
directly  judging  the  total  target  probability  and  the  probability  of 
the  most  likely  specific  target  name.  Each  of  the  three  judgment  tech¬ 
niques  proved  superior  to  multiple  regression  techniques  for  adjusting 
the  interpreter's  probability  estimates.  A  simple  linear  adjustment  of 
the  interpreter's  original  estimates  was  more  effective  than  the  complex 
multiple  regression  technique.  Of  the  three  techniques  investigated, 
the  DNV  technique  appeared  to  maximize  the  amount  of  information  which 
could  be  obtained  from  the  interpreters.  Performance  in  estimating 
probable  accuracy  deteriorated  for  all  groups  when  transferred  to  the 
free-search  task,  implying  that  training  should  probably  use  free-search 
rather  than  annotated  imagery.  Studies  to  further  enhance  the  inter¬ 
preter's  ability  to  estimate  probabilities  accurately  utilizing  the  DNV 
are  in  progress. 

The  other  basic  input  data  for  the  expected  cost  approach  are  the 
values  in  the  error  cost  matrix.  The  first  study  conducted  explored  the 
basic  feasibility  of  generating  matrices  containing  indexes  of  the  rela¬ 
tive  costs  of  errors  of  different  kinds  ( 13 ) •  The  following  objectives 
were  pursued  in  the  context  of  a  surveillance  information  processing 
system: 

1.  Explore  the  problems  in  constructing  cost  matrices,  in  partic¬ 
ular  problems  of  specifying  target  categories  for  column  and  row  headings 
and  assigning  values  to  individual  cells. 

2.  Study  the  impact  of  different  military  situations  on  the  cost 
matrices,  in  particular  the  effect  of  differences  in  the  military  situa¬ 
tion  on  intelligence  information  requirements  as  reflected  in  the  target 
categories  and  the  cost  of  various  types  of  error  (omission,  inventive, 
misidentification) . 

3.  Devise  procedures  whereby  error  cost  matrices  could  be  readily 
generated  in  the  field. 

Analysis  of  the  data  revealed  that  judged  cost  of  errors  varied  con¬ 
siderably  with  type  of  error  involved.  Errors  of  omission  were  considered 
most  serious,  misidentifications  within  a  category  of  targets  least 
serious.  Inventive  errors  and  misidentifications  across  target  categories 
were  of  intermediate  cost.  Average  costs  assigned  in  one  situation  were 
not  substantially  different  from  those  assigned  in  other  situations. 
However,  some  situational  effects  were  noted;  for  example,  the  cost  of 
errors  in  identifying  mine  fields  was  greater  in  attack  than  in  defense 


situations.  Errors  involving  certain  targets  were  consistently  judged 
more  costly  than  errors  on  other  targets.  Errors  in  identifying  tanks 
and  missiles  were  considered  the  most  serious  and  identification  of  . 
supply  facilities  the  least  serious,  regardless  of  situation. 

The  procedure  used  to  obtain  the  error  cost  estimates  was  opera¬ 
tionally  difficult  and  time-consuming,  and  the  quantitative  estimates 
obtained  were  of  low  inter- judge  reliability.  On  the  other  hand,  when 
the  officers  were  asked  to  rank  targets  subjectively  according  to  the 
positive  value  or  importance  of  information  about  them,  the  task  was 
performed  adequately  and  well,  and  inter-judge  reliability  of  the  values 
was  high.  Thus,  though  the  feasibility  of  obtaining  cost  estimates  of 
interpretation  errors  was  demonstrated,  the  close  relationship  between 
target  importance  rankings  and  the  more  elaborate  cost  estimates,  plus 
the  greater  inter- judge  reliability  of  the  rankings,  suggested  a  short¬ 
cut  method  based  on  a  ranking  procedure  which  r:ould  be  used  in  the  field 
to  generate  cost  matrices. 

A  second  study  was  conducted  to  capitalize  on  the  above  findings  and 
develop  a  practical  field  methodology  for  obtaining  error  cost  estimates 
of  improved  reliability  and  validity  (31)  •  Practical  considerations 
imposed  several  important  requirements:  1)  The  method  sought  must  be 
capable  of  providing  valid  costs  on  an  interval  scale;  2)  it  must  require 
no  more  than  four  hours  of  a  decision  maker's  time  to  establish  all  costs; 
and  3)  it  must  require  neither  special  equipment  nor  lengthy  training  of 
the  decision  maker  for  data  collection.  Direct  judgments  of  cost  for  a 
selected  set  of  different  possible  errors  and  direct  judgments  of  value 
for  correct  reports  of  every  type  of  target  were  obtained.  These  costs 
«md  values  were  then  employed  as  variables  in  a  multiple  regression 
equation  for  the  prediction  of  unjudged  error  costs. 

The  short-cut  method  for  quantifying  the  subjective  costs  of  large 
numbers  of  image  classification  errors  via  direct  magnitude  estimation 
of  a  sample  of  errors  and  the  subsequent  prediction  of  all  remaining 
errors  was  sufficiently  promising  to  warrant  further  research  efforts 
aimed  at  refining  the  method  and  evaluating  its  effectiveness  in  an 
operational  field  type  setting  for  use  in  conjunction  with  probability 
vector  estimates  provided  by  interpreters.  The  large  number  of  errors 
which  must  be  judged  or  predicted  and  the  generation  of  appropriate 
weights  for  the  multiple  regression  prediction  equation  preclude  this 
approach  where  computer  assistance  is  unavailable.  However,  with  com¬ 
puters  comparable  to  those  in  existing  or  planned  image  interpretation 
facilities,  the  task  is  easily  manageable. 


Computer  Techniques  for  Training  Image  Interpreters 

Sensor  systems  and  aerial  platforms  are  being  continually  improved 
by  the  military  services  to  obtain  better  surveillance  cover.  With  each 
advance  in  sensor/platform  capability,  the  need  for  a  continuous  train¬ 
ing  program  designed  to  maintain  and  enhance  the  proficiency  of  inter¬ 
preters  becomes  more  imperative.  The  Information  Systems  Laboratory 


-  26  - 


provides  an  excellent  vehicle  for  conducting  research  to  evaluate  new 
techniques  for  on- job  training  of  image  interpreters.  The  advanced  image 
interpretation  facilities  being  developed  by  the  Services  contain,  like 
the  Laboratory,  various  viewing  and  input/output  devices  and  one  or  more 
computers  with  associated  data  bases.  Such  facilities  can  be  used  as 
automated  teaching  machines  to  implement  programs  for  the  maintenance 
and  improvement  of  the  proficiency  of  personnel  assigned  to  the  facilities. 
A  summary  of  completed  and  on-going  research  on  a  number  of  promising 
training  techniques  is  presented  below. 

One  study  is  concerned  with  comparing  branching  and  linear  instruc¬ 
tional  programs  for  training  in  target  identification  (32) • A  branching 
program  permits  the  presentation  of  instructional  material  to  vary  de¬ 
pending  on  the  progress  of  the  student  through  the  course.  A  linear 
program  presents  the  instructional  material  in  a  fixed  order  regardless 
of  demonstrated  mastery.  Response- sensitive  and  response- insensitive 
feedback  were  also  compared.  Response-sensitive  feedback  provides  the 
student  with  information  appropriate  to  the  specific  answer  he  has  given. 

If  he  gives  a  wrong  answer,  the  feedback  explains  why  his  answer  is  wrong. 
Response-insensitive  feedback  provides  only  the  correct  answer.  Both 
these  comparisons  have  implications  for  the  computer  capacity  and  soft¬ 
ware  complexity  necessary  for  computer-aided  instruction  (CAl)  systems, 
the  branching  and  response-sensitive  methods  requiring  more  support  than 
the  linear  and  response-insensitive  methods.  Considerable  learning  was 
accomplished  through  the  medium  of  computer-assisted  instruction.  The 
findings  also  indicated  that  the  instructional  approaches  requiring 
greater  computer  capacity — response-sensitive  feedback  and  branching 
programs--may  yield  more  efficient  learning.  The  possible  use  of  the  com¬ 
puter  within  an  advanced  tactical  image  interpretation  facility  for  main¬ 
taining  and  enhancing  personnel  proficiency  should  be  kept  in  mind  in 
the  selection  and  employment  of  computer  and  associated  input /output 
equipment . 

In  a  rapidly  changing  environment,  CAI,  as  usually  applied,  has  the 
disadvantage  of  requiring  considerable  time  for  course  preparation. 

Material  to  be  learned  must  be  carefully  formatted  and  programmed  for 
presentation  in  discrete  knowledge  bits.  Try-outs  of  the  instructional 
units  are  necessary  to  insure  effective  learning.  Another  approach  to 
CAI,  applicable  where  a  large  data  bank  is  associated  with  a  computer, 
is  to  allow  the  student  to  interact  directly  with  the  computer  without 
the  mediation  of  a  course  author.  Material  to  be  learned  is  presumed  to 
be  present  in  the  system  data  bank  and  is  accessed  through  use  of  the 
data  bank  index.  An  advantage  over  conventional  CAI  is  that,  as  soon  as 
information  becomes  part  of  the  data  bank,  it  can  become  incorporated 
into  a  programmed  learning  routine.  Two  on-going  studies  are  exploring 
how  this  modified  approach  to  CAI  may  be  accomplished  in  an  advanced 
image  interpretation  facility. 

In  the  first  study,  an  open-ended  general  purpose  image  interpreter 
Computer  Aided  Instruction  (CAl)  program  will  be  designed  in  which  target 
types  will  not  be  broken  down  into  component  features  but  the  interpreter 


will  be  allowed  to  make  hi  discriminations  based  on  whatever  cues  he 
can  discover  in  the  material  presented.  It  will  be  assumed  that  the 
image  interpretation  system  data  bank  contains  examples  of  the  targets 
to  be  learned.  The  general  purpose  program  uses  the  data  bank  index  to 
find  target  examples,  assigns  numbers  to  each  target  example,  controls 
the  presentation  of  the  examples,  monitors  students'  answers,  and 
supplies  appropriate  feedback.  The  principle  guiding  the  presentation 
of  target  examples  is  to  present  to  the  student  at  any  given  time  a 
target  ei  umple  selected  randomly  from  those  targets  and  examples  that 
the  student  is  equally  likely  not  to  be  able  to  correctly  identify. 

When  the  student  has  demonstrated  that  he  can  correctly  identify  each 
target  a  predetermined  number  of  times,  the  course  is  over.  A  gener¬ 
alized  flow  diagram  of  this  instructional  strategy  is  presented  in 
Figure  6.  In  the  experimental  evaluation,  this  approach  will  be  con¬ 
trasted  with  the  more  conventional  CAI  programs  developed  in  the  pre¬ 
viously  described  study. 

In  the  second  study,  a  more  conventional  structured  course  on  the 
interpretation  of  infrared  (IR)  imagery  will  be  contrasted  with  an 
approach  which  can  perhaps  best  be  described  as  "browsing."  In  browsing, 
the  student  searches  out  through  the  data  bank  index  and  studies  material 
concerning  sensor/target  characteristics.  The  value  of  introducing 
practical  exercises  into  the  learning  process  will  also  be  explored. 

The  amount  of  computer  software  and  CAI  authorship  involved  in  implement¬ 
ing  the  browsing  approach  is  minimal.  Whether  it  is  an  effective  train¬ 
ing  technique  will  be  determined  empirically. 

The  CAI  methods  described  above  are  directed  primarily  toward  en¬ 
hancing  image  interpreters'  identification  accuracy  and  completeness. 
Another  experiment  was  designed  to  assess  techniques  for  increasing  image 
interpreters'  detection  speed  while  maintaining  initial  accuracy  and  com¬ 
pleteness  levels  (33)  •  Two  search  strategies  devised  to  reduce  ineffi¬ 
cient  search  behavior  while  increasing  the  completeness  with  which  images 
life'searched  were  coupled  with  a  "speed-reading"  technique  for  reducing 
visual  fixation  times  and  expanding  the  effective  size  of  interpreters' 
perceptual  field.  One  search  strategy  employed  a  systematic  geometrical 
scan  pattern  in  which  the  subjects  searched  across  the  image  very  much 
as  they  would  a  page  in  English — from  top  to  bottom,  scanning  from  left 
to  right.  The  other  search  strategy  was  tactically  oriented.  The 
students  were  taught  to  allow  their  search  to  be  guided  by  the  signifi¬ 
cant  tactical  features  and  lines  of  cunmunication  present  in  the  image — 
ridges,  valleys,  roads,  rivers,  trails,  for  example.  These  two  search 
strategies  were  compared  with  each  other  and  with  a  free-search  strategy 
in  which  the  students  were  not  trained  to  search  the  imagery  in  any 
particular  fashion.  They  were  not  given  special  visual  fixation/field 
expansion  training,  but  were  given  special  practice  and  feedback  designed 
to  increase  search  speed.  Results  indicate  that  interpreters  can  be 
trained  to  increase  their  speed  considerably  with  little  loss  in  accuracy 
and  completeness.  This  desirable  result  was  achieved  simply  by  forcing 
interpreters  to  gradually  decrease  their  search  time  per  image.  Further, 
completeness  rates  can  be  maintained  while  increasing  speed  through 
training  subjects  to  scan  images  systematically.  If  the  concomitant 
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Figure  6.  Generalized  flow  diagram  for  data  bank  utilization 
Computer-Aided  Instruction  (CAI) 
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tendency  to  make  more  inventive  errors  can  be  eliminated,  training  in 
the  use  of  a  gepmetric  search  pattern  may  prove  an  effective  procedure 
also.  In  fact,  reducing  the  number  of  inventive  error  responses  would 
raise  detection  accuracy  and  most  probably  increase  speed  for  all  methods. 

, Follow-up  research  applying  che  geometric  and  speed  techniques  to  search¬ 
ing  real-time  or  moving  imagery  is  currently  under  way. 

Another  study  concerned  error  avoidance  training.  Interpreters  can 
make  the  error  of  reporting  targets  where  none  really  exist.  The  fre¬ 
quency  of  these  inventive  errors  depends  on  a  number  of  factors--the 
scale,  quality,  and  complexity  of  the  imagery,  the  number  of  natural  and 
man-made  objects  in  the  imagery  that  can  be  confused  with  military  tar¬ 
gets,  and  the  experience  and  training  of  the  interpreters.  The  instruc¬ 
tional  unit  developed  is  based  on  the  rationale  that  through  studying 
the  kinds  of  error  made  by  interpreters  who  have  previously  examined 
similar  imagery,  interpreters  can  learn  to  avoid  making  the  same  errors 
(33).  Error  avoidance  training  proved  to  be  an  effective  technique  for 
reducing  the  number  of  inventive  errors  made  by  interpreters.  An  advan¬ 
tage  of  the  technique  is  that  it  can  be  applied  in  the  classroom  as  well 
as  in  the  field,  using  examples  specific  to  particular  unit  operations. 

A  final  study  concerns  team  consensus  feedback  (34).  Results  from 
a  number  of  studies  conducted  at  BESRL  have  indicated  that  image  inter¬ 
preters  working  in  teams  can  produce  more  complete  and  accurate  informa¬ 
tion  than  interpreters  working  alone.  The  consensual  judgment  of  team 
members  has  been  found  to  be  especially  effective  in  reducing  the  number 
of  identification  errors  made  by  single  interpreters.  These  results  led 
to  the  development  of  experimental  training  methods  based  on  teams'  pro¬ 
viding  feedback  to  the  individual  members  comprising  the  teams.  The 
identifications  agreed  upon  by  team  members,  being  on  the  whole  more 
accurate  and  complete  than  those  of  the  average  individual  team  member, 
can  provide  feedback  to  the  individual  team  member  as  to  the  adequacy  of 
his  own  identifications.  An  advantage  of  this  approach  is  that  no 
special  instructional  materials  or  preparation  are  required.  Teams  can 
be  formed  in  the  field  on  an  ad  hoc  basis  during  operational  lulls.  Any 
previously  interpreted  imagery  that  happens  to  be  available  can  be  used. 

A  series  of  four  experiments  was  conducted  to  evaluate  the  effec¬ 
tiveness  of  training  methods  based  on  team  consensus  feedback.  In  the 
last  experiment,  the  two  most  promising  techniques  identified  in  earlier 
research,  the  serial  consensus  and  immediate  consensus  methods,  were  com¬ 
pared  with  each  other  and  with  a  control  procedure  (individuals  practicing 
alone).  In  the  serial  consensus  method  each  man  performs  initial  inter¬ 
pretation  on  a  different  stereo  pair.  The  men  then  move  to  their  team- 
mates'  positions,  check  identifications  already  made,  and  add  any  addi¬ 
tional  targets  detected.  When  each  man  has  checked  the  stereo  pairs, 
the  men  get  together  and  discuss  any  conflicts.  Under  the  immediate 
consensus  method,  each  man  performs  initial  interpretation  on  a  copy  of 
the  same  stereo  pair.  The  men  get  together  and  discuss  all  conflicts. 

Under  the  control  procedure,  each  man  interpreted  the  imagery  by  himself 
and  did  not  discuss  or  compare  response  data  sheets  with  other  interpreters 


Team  consensus  feedback  methods  gave  significant  performance  increments 
over  individuals  practicing  alone.  Differences  between  the  serial  con¬ 
sensus  and  immediate  consensus  methods  were  negligible.  The  personnel 
benefiting  most  from  the  training  were  the  relatively  low  performance 
interpreters.  Most  field  interpretation  units  typically  have  a  rela¬ 
tively  large  number  of  inexperienced  personnel  and  a  relatively  small 
number  of  experienced  personnel.  A  program  in  which  recent  interpreta¬ 
tion  school  graduates  and  transferees  can  be  given  systematic  practice 
exercises  during  slack  periods  offers  promise  for  improving  personnel 
proficiency. 

In  summary,  a  number  of  image  interpreter  training  techniques  are 
being  investigated  at  BGSRL.  The  experimental  findings  obtained  to  date 
indicate  that  three  of  the  techniques  show  sufficient  promise  to  warrant 
at  least  tentative  inclusion  in  programs  to  maintain  and  enhance  person¬ 
nel  proficiency  in  the  field  and  perhaps  in  the  classroom  as  well.  The 
speed  expansion  technique  of  simply  encouraging  interpreters  to  system¬ 
atically  increase  their  speed,  giving  them  time  feedback  and  plenty  of 
practice,  seems  effective  in  improving  the  rate  of  image  interpretation. 
The  error  key  approach  and  practice  in  error  avoidance  seem  effective  in 
reducing  the  number  of  inventive  errors.  Team  consensus  training  seems 
effective  in  increasing  detection  and  identification  performance.  All 
these  techniques  involve  practice.  As  a  matter  of  fact,  the  data  support 
the  contention  that  no  matter  what  technique  is  used,  increments  in  per¬ 
formance  will  result  from  practice--though  some  techniques  produce 
greater  increments  than  others.  To  date,  no  longitudinal  data  have  been 
collected  to  study  the  effect  of  prolonged  intermittent  practice  over 
periods  lasting  several  months  or  years.  However,  with  the  emergence  of 
new  sensors,  new  targets,  and  new  military  situations,  the  skills  of 
image  interpreters  will  clearly  need  to  be  continually  updated.  The 
methods  recommended  above  lend  themselves  readily  to  a  long-term  program 
of  proficiency  maintenance. 

The  case  for  computer-assisted  instruction  is  perhaps  not  substan¬ 
tiated  as  yet,  although  the  results  of  the  initial  experiment  in  this 
area  were  certainly  encouraging.  Advanced  image  interpretation  facili¬ 
ties  with  their  computers,  data  base,  .displays,  and  input/output  devices 
provide  ready-made  beds  for  training.  Hopefully,  data  from  current  and 
planned  studies  will  indicate  how  these  potentially  powerful  tools  can 
best  be  used  to  enhance  interpreter  proficiency  in  the  field.  The 
potential  application  of  CAI  in  the  classroom  will  also  be  explored  in 
future  research.  To  this  end,  BESRL  is  planning  in  cooperation  with  the 
U.  S.  Army  Intelligence  School  to  place  two  terminal  on-line  stations  at 
the  School  to  conduct  CAI  research. 
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THE  INFORMATION  SYSTEMS  LABORATORY 


The  BESRL  Information  Systems  Laboratory  (Figure  A-I)  provides  a 
highly  flexible  controlled  environment  with  the  capability  of  human  per¬ 
formance  experimentation  and  evaluation  ranging  from  that  of  the  indi¬ 
vidual  interpreters  and  associated  equipment  and  methodological  aids  to 
that  of  a  total  semiautomated  computerized  information  processing  system. 
The  total  range  and  versatility  of  the  experimentation  is  limited  only 
by  the  state  of  the  art  in  that  equipment  is  modular  in  nature  and  can 
be  configured  and  augmented  to  meet  a  multitude  of  requirements.  The 
heart  of  the  laboratory  is  the  computer  system,  which  includes  the  com¬ 
puter  itself  and  peripheral  equipment  which  drives  or  controls  the  on¬ 
line  laboratory  equipment.  In  addition,  punched  paper  tape  equipment 
and  typewriter  printers  are  provided  for  off-line  or  noncomputer  based 
experimentation. 


THE  COMPUTER  SYSTEM 

The  computer  is  a  medium-scale  general-purpose  computer  (CDC  3300)^ 
which  is  time-shared  for  batch  job  statistical  processing  at  the  same 
time  an  experiment  is  in  progress.  The  computer  peripherals  include  a 
card  reader,  a  card  punch,  a  line  printer,  two  magnetic  tape  units, 
three  disc  drives,  a  controller  for  driving  the  cathode  ray  tubes,  and 
a  controller  for  driving  the  other  on-line  laboratory  equipments. 

The  computer  system  is  depicted  graphically  in  Figure  A-2.  The 
Central  Processor  (3304)^-'  is  the  controlling  unit  comprising  arithmetic 
control,  input /output  control,  three  index  registers,  a  real-time  clock, 
an  operator's  console,  and  an  input/output  typewriter.  It  performs  both 
word  and  character  manipulations.  The  Storage  Modules  (3302)  are  core 
memory  storage  units  which  each  contain  16,384  words.  Access  time  for 
one  word  in  storage  is  1.25  microseconds.  Each  of  the  four  Data  Channels 
(3306)  is  bi-directional  and  can  transfer  12  bits  in  parallel.  Each 
channel  can  operate  asynchronously  and  simultaneously  with  each  other 
and  with  the  central  processor.  Either  data  channel  may  connect  to  the 
two  disk  controllers.  The  Disk  Pack  Controller  (3290C)  acts  as  a  commu¬ 
nications  coordinator  between  the  computer  and  the  display  CRTs.  It 
provides  display  buffer  memory,  symbol  generation  and  refreshing,  and 
interface  with  the  computer.  It  can  control  up  to  twelve  CRTs  and 
transfer  data  at  a  rate  of  up  to  ^0,000  characters  per  second. 


^  Commercial  designations  are  given  only  in  the  interest  of  precise  de¬ 
scription;  their  mention  does  not  constitute  indorsement  by  BESRL  or 
by  the  Army. 
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The  unbuffered  Card  Reader  Controller  (3248)  provides  Interface  con¬ 
trol  for  the  Card  Reader  (405).  The  Card  Reader  operates  at  1200  cards 
per  minute  and  can  read  Hollerith  or  binary-coded  punched  cards.  The 
Card  Punch  Controller  (3245)  furnishes  intercommunication  control  between 
the  Card  Punch  (523)  and  the  central  processor.  Hollerith  or  binary-code 
cards  can  be  punched  at  ranges  up  to  100  per  minute.  The  Line  Printer 
(505),  prints  500  lines  a  minute,  has  a  print  span  of  120  characters,  and 
prints  six  lines  per  inch.  A  line  buffer  and  controller  are  also  fur¬ 
nished. 

The  line  controllers  provide  a  total  of  22  subchannels  for  communi¬ 
cation  between  the  laboratory  stations  and  the  central  processor.  Each 
station  can  be  modularly  equipped  with  a  variety  of  devices  and  served 
by  one,  two,  or  more  subchannels. 


LABORATORY  EQUIPMENT 

In  addition  to  six  automated  light  tables,  the  laboratory  is  equipped 
with  a  variety  of  lisplay  and  input/output  devices  that  can  be  connected 
on  line  with  the  computer  as  shown  in  Figure  A-3.  All  laboratory  equip¬ 
ment  except  the  CRT  devices  operates  through  a  1051  control  unit.  The 
Control  Unit  controls  all  data  flow  into,  through,  and  out  of  the  1050 
system.  The  system  provides  for  home-loop  operation  (transmitting  to 
other  units  at  the  same  laboratory  station)  or  line-mode  transmission  to 
the  computer  and  to  other  stations  in  the  laboratory. 

Au  1  .ed  Light  Table.  These  units  are  basically  table-like  devices 
with  a  translucent  top,  a  film  transport  mechanism,  film  take  up  spools 
at  either  end  and  a  variable  light  source  underneath  the  translucent 
panel.  A  close-up  view  of  one  of  these  tables  is  shown  in  Figure  A-4. 

They  have  several  unique  features  for  aiding  the  interpreter  in  his  task 
of  extracting  information  from  a  film  record  and  for  facilitating  the 
conduct  of  experiments.  First  and  perhaps  foremost  is  the  fact  that  the 
light  tables  can  communicate  with  the  computer  and  thus  can  be  considered 
an  input/output  device.  Computer  programs  can  be  written  which  enable 
the  computer  to  control  the  light  table,  and  at  the  same  time  the  com¬ 
puter  can  recognize  light  table  functions  that  have  been  initiated  by  an 
operator.  Computer  control  can  cause  selected  indicator  lights  at  the 
operator's  console  to  go  on  or  off,  activate  an  audible  alert  at  the 
light  table,  and  regulate  the  rate  and  mode  (continuous  or  stop-start) 
of  film  movement  across  the  light  table. 

Specific  responses  can  be  communicated  from  the  operator  to  the  com¬ 
puter  by  depressing  appropriate  buttons  on  a  40-button  response  panel. 
Target  location  in  terms  of  XY  coordinate  can  be  transmitted  by  position¬ 
ing  the  reticle  of  the  table  tube  magnifier  over  the  target  on  the  film 
and  activating  a  foot  switch.  Frame  marks  passing  under  a  photo-electric 
sensor  generate  signals  which  enable  the  computer  to  recognize  film  move¬ 
ment,  direction  and  number  of  frames. 
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Film  speed  is  infinitely  variable  between  .1  inch  per  second  and  5 
inches  per  second.  As  many  as  4  different  speeds  can  be  preset  and  then 
automatically  selected  or  reselected  during  the  experiment.  Precise 
timing  of  subject  responses  via  the  millisecond  clock  internal  to  the 
computer,  and  immediate  statistical  processing  of  responses  can  be 
achieved  when  the  tables  are  operated  on-line  with  the  computer. 


Figure  A-4.  The  ISL  automated  light  table  with  associated  controls, 
register,  and  push  button  response  panel 


CRT  Input /Output  Display  Device.  As  many  as  12  of  the  electronic 
typewriter/display  devices  that  allow  two-way  communication  with  the 
computer  can  be  utilized  in  the  laboratory.  These  devices  consist  of  a 
typewriter  kayboard  and  a  bright  CRT  display  screen.  Each  display  page 
is  20  lines  of  %  characters  per  line.  The  symbol  repertoire  consists 
of  63  characters.  A  controllable  index  marker  indicates  where  the  next 
character  to  be  typed  will  appear.  The  screen  may  be  filled  with  data 
by  the  operator  and  these  data  transmitted  to  the  computer  for  process¬ 
ing,  or  the  computer  may  fill  the  screen  with  data  to  be  viewed  by  and 
acted  upon  by  the  operator.  Transmitting  to  or  from  the  computer  re¬ 
quires  only  40  milliseconds,  so  that  to  the  subject  it  appears  to  be 
instantaneous . 
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Closed-Booth  Image  Inter 


station  Station.  There  are  three  closed- 


booth  Image  interpretation  stations.  Typical  of  these  is  the  one  shown 
in  Figure  A-5.  These  stations  provide  essentially  the  same  environ¬ 
mental  support  as  the  automated  light  tables  but  with  a  number  of  differ¬ 
ences.  The  closed  booth  provides  control  over  both  sound  and  ambient 
light  levels,  so  that  the  subject  may  be  isolated  from  disturbing  influ¬ 
ences.  The  stations  have  rear  view  projectors  with  variable  light 
intensity  and  magnification  of  the  image  under  subject  control.  This 
equipment  gives  the  subject  "whole  image"  magnification  up  to  4  times 
for  5-inch  film  instead  of  the  "spot"  magnification  obtained  when  a  tube 
magnifier  is  used  on  the  automated  light'  table.  Spot  or  area  magnifica¬ 
tion  up  to  10  times  is  also  possible  with  these  devices.  The  subject 
selects  the  portion  of  the  original  frame  that  he  wishes  to  view  by 
translating  in  both  the  x  and  y  directions.  Rotation  of  the  image  up  to 
560°  is  also  under  subject  control  so  that  the  subject  may  orient  the 
image  suitably.  When  a  subject  moves  the  image  under  the  cursor  or  the 
cursor  over  a  spot  on  the  image,  the  type  and  amount  of  motion  can  be 
transmitted  to  the  computer  so  that  x-y  coordinate  location  and  measure¬ 
ments  can  be  calculated.  Indicator/response  buttons  are  available  for 
two-way  communication  with  the  computer.  Auxilliary  input/out  devices 
such  as  the  CRT  can  be  located  in  the  booth.  The  1052  printer-keyboard 
is  shown  in  the  illustration. 


Figure  A-5.  A  closed  booth  station  showing  the  full  frame  magnifi¬ 
cation  screen,  controls,  response  push  buttons  and 
I/O  typewriter-printer 


t 
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Random  Access  Slide  Projector.  Four  70mm  and  two  35™  random  access 
slide  projectors  can  be  directed  by  the  computer  or  independently  to  dis¬ 
play  any  slide  in  their  carousels  (sixty  70mm  slides  or  one  hundred  3*5™ 
slides).  The  time  required  to  change  from  one  slide  to  another  depends 
on  the  distance  between  the  two  slides  in  the  carousel  and  ranges  from 
two  to  eight  seconds.  Movable  iris  lens  and  variable  intensity  of  the 
projection  lamp  enable  refined  control  of  the  projected  image.  The 
slides  are  normally  projected  onto  rear-projection  viewing  screens  for 
simulation  of  a  variety  of  proposed  information  displays.  Non-computer 
control  can  be  effected  remotely  if  desired. 

Printer  Keyboard  (1052).  There  are  five  of  these  devices,  each  con¬ 
taining  a  printing  mechanism  and  a  typewriter  keyboard.  The  printer  can 
accept  data  from  the  computer,  another  station,  the  paper  tape  reader, 
or  its  own  keyboard.  The  keyboard  is  also  used  to  input  data.  The 
operator  can  type  at  any  rate  he  chooses.  Transmission  is  at  the  rate 
of  15  characters  per  second. 

Paper  Tape  Punch  ( 1055) .  The  five  units  in  BESRL's  Information 
Systems  Laboratory  can  record  characters  generated  by  the  computer,  the 
light  table,  the  keyboard  printer,  or  the  programmed  keyboard,  providing 
a  means  of  recording  subject  responses  without  the  use  of  a  computer. 

The  information  on  the  paper  tape  can  be  typed  out  or  processed  by  the 
computer  at  a  later  time. 


Paper  Tape  Reader  (1054).  The  paper  tape  reader  can  read  and  trans¬ 
mit  information  which  was  previously  punched  on  paper  tape.  The  reader 
can  operate  the  keyboard  printer,  the  light  table,  and  any  of  the  other 
laboratory  devices  which  can  be  operated  by  the  computer  with  the  excep¬ 
tion  of  the  CRTs.  For  example,  a  test  of  the  light  table  can  be  placed 
on  paper  tape  and  used  daily,  or  information  to  be  presented  to  a  subject 
can  be  placed  on  paper  tape  for  use  in  a  non- computer-based  experiment. 

Programmed  Keyboard  (1092).  The  six  keyboards  in  the  Information 
Systems  Laboratory  contain  a  matrix  of  buttons,  with  ten  buttons  in  each 
column  and  sixteen  columns.  A  keymat  can  be  placed  over  the  keyboard  to 
show  the  meaning  of  each  button.  The  keyboards  can  be  used  to  transmit 
information  to,  or  request  information  from,  the  computer  or  other 
devices.  When  the  subject  has  composed  a  message  by  depressing  selected 
buttons  on  the  keyboard,  the  contents  of  the  keyboard  are  transmitted  to 
the  computer  by  depressing  a  special  button.  Suppose  this  keyboard  is 
to  be  used  to  identify  a  target.  The  first  column  of  buttons  might  des¬ 
ignate  target  type,  the  second  column  the  status  of  the  target  (moving, 
firing  position,  etc.),  the  third  column  the  location,  etc.  The  subject 
enters  his  selections  by  depressing  the  appropriate  buttons,  checks  his 
entries  visually,  then  pushes  a  log  button  to  transmit  the  information. 

In  the  case  of  requesting  information,  a  display  device  (CRT  or  printer) 
would  be  needed  to  record  the  returning  information. 


APPENDIX  B 


SYSTEMS  MEASUREMENT  METHODOLOGY 


CONSTRUCTING  THE  SSMB 

Selection  of  the  image  interpretation  functions  to  be  represented 
was  based  on  a  previous  studj^  wherein  a  concept  for  imagery  interpreta¬ 
tion  in  an  advanced  TIIF  was  evolved  from  functional  task  analysis,  data 
on  human  and  machine  performance,  and  data  on  the  frequency  of  task 
occurrence.  The  major  functions,  subfunctions,  and  tasks  identified 
were  revised  to  arrive  at  a  more  generalized  list,  excluding  those 
unique  to  a  specific  configuration,  since  it  is  Intended  that  the  SSMB 
be  equally  applicable  to  a  variety  of  configurations.  Table  B-l  is  a 
listing  of  the  functions  and  subfunctions  represented. 

The  echelon  of  operation  to  be  simulated  was  determined  to  be  corps 
level,  primarily  on  the  basis  that  this  is  the  lowest  level  under  current 
Army  doctrine  having  organic  aircraft.  The  anticipated  capability  of  a 
TIIF  is  such  that  it  was  felt  neither  the  full  diversity  of  activities 
and  requirements  or  a  sufficiently  taxing  workload  could  be  realistically 
imposed  in  lower  echelons  not  having  organically  assigned  aircraft. 
Further  recourse  to  standard  Army  doctrine  regarding  imagery  interpreta¬ 
tion  requirements  and  operations  at  corps  level  provided  the  basis  for 
determining  the  appropriate  types  of  target,  activities  to  be  interpreted, 
reporting  requirements,  and  relationship  of  the  image  interpretation 
facility  to  G2  Air  and  the  command  organization. 

Guided  by  the  above  determinations,  the  desired  test  characteristics 
were  specified  for  the  major  test  components- -inputs,  requirements,  and 
outputs  as  shown  in  Table  B-2.  Overlaying  all  these  characteristics  was 
the  additional  requirement  for  a  realistic  and  coherent  scenario  that 
would  effectively  bond  together  all  elements  of  the  test  and  convey  to 
the  image  interpreter  both  explicitly  and  implicitly  the  performance 
demands  of  the  situation  depicted. 

Imagery  was  selected,  SOP's  and  reporting  requirements  established 
and  supporting  documentation  selected  and/or  prepared--scenario,  report 
forms,  Essential  Elements  of  Information,  Standing  Requirements  List, 
Mission  Requests,  Mission  Plans,  Pilots  Debriefs,  Maps,  Pilots  Flight 
Traces,  Special  Requests  for  Information,  Photo  Intelligence  Data  Cards, 
and  school  solution.  Concurrently,  data  collection  and  processing  plans 
were  formulated  which  would  allow  evaluation  of  total  system  effective¬ 
ness  and  the  relative  contribution  of  various  system  components  to  total 
performance  in  terms  of  accuracy,  completeness,  and  time.  In  addition, 
provision  was  made  for  obtaining  directly,  or  deriving,  appropriate 


^Hansen,  0.  K.,  K.  R.  Colson,  and  L.  Vettese  (Nortronics)  and  H.  C. 
Strasel  (BESRL).  Method  for  developing  a  laboratory  model  of  an  image 
interpretation  system  (Unclassified  Title). 
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measures  of  performance  for  some  of  the  internal  operations  and  proce¬ 
dures  and  the  behavior  of  personnel  which  could  conceivably  affect  system 
performance.  All  materials  were  organized  and  integrated ,  and  input 
schedules  established.  Instructional  booklets  were  prepared  to  assist 
interpreters  in  understanding  the  nature  and  purpose  of  the  exercise  and 
the  requirements  to  be  imposed.  Test  management  handbooks  were  developed 
to  provide  specific  guidance  to  experimenters  regarding  the  administra¬ 
tion  of  the  test,  the  sequence  of  events,  and  data  collection  and  pro¬ 
cessing  procedures. 


Table  B-l 

^UNCTIONS  AND  SUBFUNCTIONS  TO  BE  REPRESENTED  IN  THE 
STANDARD  SYSTEMS  TEST 


FUNCTION 

SUBFUNCTION 

Pre-mission  Notification 

Receive  Mission  Plan 

and  Preparation 

(Does  not  include  SRl's)  (6  tasks^ 
Pre-mission  Preparation  (9  tasks) 

Specific  Request  for 

Receive  and  Evaluate  Specific  Request 

Information  Processing  (SRI) 

for  Information  (SRI)  (4  tasks) 

Answer  SRI  from  Available  Imagery 
(7  tasks) 

Preparation  for  Imagery 

Receive  Mission  Imagery 

Analysis 

(hand  delivered)  (6  tasks) 

Prepare  for  Interpretation  (8  tasks) 

Analyze  and  Interpret  Imagery 

Analyze  some  Imagery  for  Hot  Report 
when  specifically  required  ( 17  tasks) 

Analyze  Photo  and  IR  coverage  for 
Immediate  or  Detailed  Report  (27  tasks) 

Analyze  SLAR  Coverage  (7  tasks) 

Reporting 

Prepare  Hot  or  Immediate  Report 
(3'  tasks) 

Prepare  Detailed  or  Special  Reports 
(3  tasks) 

Post  Mission  Operations 

Return  Internal  Reference  Material  to 
Storage  (4  tasks) 
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Table  B-2 


DESIRED  TEST  CHARACTERISTICS 


TESTS  COMPONENT 


Inputs 


Requirements 


Outputs 


DESIRED  CHARACTERISTICS 

Comparative  cover  of  contemporary  military 
targets  and  situations 

Multi-sensor  imagery  and  formats 

Compatible  with  corps  level  of  II  operations 

Ground  truth  or  school  solution  ootainable 

Map  coverage  and  supporting  documentation 
available 

Sufficient  quantity  of  imagery  to  provide  for 
several  days  of  training  and  testing 

Security  classification  mostly  unclassified  and 
in  no  event  higher  than  confidential 


Clear  statement  of  image  interpretation 
processes  to  be  performed 

Elicit  performance  of  representative  functions 

Compatible  with  corps  level  operations 

Must  be  generalizable  across  a  variety  of 
systems 

Must  evoke  objectively  scorable  responses 


Representative  of  Army  operational  reports 

Scorable  in  terms  of  timeliness,  accuracy,  and 
completeness 

Comparable  to  school  solution  in  form  and 
content 

Sufficient  quantity  to  produce  reliable  results 
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MODEL  CONCEPTUALIZATION 


Steps  in  the  model  development  include  1)  analysis  o£  TIIF-type 
function!  and  tasks,  2)  determination  of  manned-position  types  for  task 
performance,  3)  investigation  of  manual  vs  semiautomated  options  for 
certain  subfunctions,  and  4)  investigation  of  the  suitability  of  the 
model  to  meet  the  requirements  for  a  system  at  the  corps  level  of  tacti¬ 
cal  usage.  Particular  attention  was  given  the  sharing  of  systems  work 
between  the  man  and  th£  machine.  The  man  was  to  be  assisted  and  guided, 
but  he  was  not  to  be  driven  by  the  automated  subsystems  in  the  accom¬ 
plishment  of  his  tasks.  The  result  of  these  analyses  was  a  conceptual 
model  of  the  image  Interpretation  system  in  terms  of  the  processes, 
personnel,  and  equipment  within  the  structural  framework  of  an  expan¬ 
sible  van. 

The  model  TIIF  specifies  six  image  interpretation  stations  which 
can  accommodate  up  to  eight  interpreters.  The  floor  plan  is  presented 
in  Figure  3  In  the  text.  Stations  A  and  B  are  configured  for  rear  pro¬ 
jection  viewing  of  Imagery  and  selected  references.  Station  A  is  de¬ 
signed  to  be  capable  of  near  real-time  viewing  and  interpretation  by 
processing  and  projecting  imagery  transmitted  from  the  reconnaissance 
aircraft  via  a  telemetry  data  link.  Stations  C,  D, and  E  provide  light- 
table  viewing  of  image  transparencies  from  all  standard  reconnaissance 
sensors.  The  five  stations  (A  through  E)  are  connected  to  the  computer 
through  individual  controls.  Station  F  is  a  manually-operated  back¬ 
lighted  drafting  table  that  is  used  for  plotting  and  for  comparison 
viewing  of  imagery  in  the  larger  formats,  such  as  two  9~inch  film  rolls. 
Station  G  is  a  communications  teletypewriter. 

For  field  use,  the  model  TIIF  was  designed  to  fit  into  an  expansible 
van  that  was  7%  feet  wide  by  17  feet  long  in  transport,  and  13%  by  17 
feet  overall  when  opened.  To  conform  to  these  dimensions,  the  fixed 
equipment  was  arranged  along  the  major  axis  of  the  van.  The  movable 
sides  were  kept  free  of  equipment  except  for  folding  shelves  and  seats, 
so  that  the  wall  space  can  be  used  to  display  maps  and  plots.  Because 
of  the  limited  space,  a  small  computer  will  be  programmed  to  perform 
limited  support  functions  such  as  translating  digitally  recorded  flight 
data,  retrieving  and  displaying  references,  managing  system  files,  com¬ 
posing  reports,  performing  routine  computations  and  communicating  with 
other  systems.  The  image  interpretation  and  system  management  operations 
are  to  be  performed  by  the  intelligence  personnel  generally  assigned  to 
such  a  facility. 

Simulation  of  the  Model.  Prior  to  constructing  the  simulation  in 
the  laboratory,  the  following  guidelines  were  established: 

1.  The  simulation  was  to  be  composed  of  available  laboratory 
equipment . 

2.  The  physical  dimensions  and  configuration  of  the  individual, 
station  equipment  were  to  duplicate  as  closely  as  possible 
those  of  the  model . 
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J.  The  simulation  was  to  be  given  the  capability  of  performing 
all  the  functions  required  by  the  SSMB. 

4.  Functions  not  required  for  the  system  test  were  to  be  only 
hypothecated  in  the  simulation. 

5.  The  computer  was  to  be  programmed  to  perform  only  the  support 
operations  envisioned  for  the  actual  TIIF  in  the  field. 

6.  The  excess  computer  capability  was  to  be  used  to  simulate 
functions  not  satisfactorily  provided  by  the  laboratory 
equipment  and  to  collect  test  data. 

7.  The  operating  procedures  for  the  simulation  were  to  reflect 
current  Army  doctrine  regarding  image  interpretation  opera¬ 
tions  in  the  field. 

These  guidelines  were  particularly  useful  in  controlling  the  tendency  to 
endow  the  simulation  with  capabilities  that  the  fieldable  TIIF  would  not 
have. 


The  floor  plan.  Figure  4  in  the  text,  illustrates  how  closely  the 
laboratory  simulation  resembles  the  conceptualized  model.  However, 
several  compromises  had  to  be  made,  primarily  to  accommodate  the  size 
and  bulk  of  the  laboratory  equipment  at  each  station.  None  of  these 
compromises  appear  to  be  of  the  type  which  would  seriously  alter  the 
functional  representation.  Fortunately,  the  inventory  of  laboratory 
equipment  included  materials  which  can  very  satisfactorily  duplicate 
the  primary  functional  systems  needed  at  each  station.  Furthermore, 
the  GDC  3300  computer2-' capabilities  not  only  met,  but  exceeded,  those 
needed  to  duplicate  the  support  processes  of  the  much  smaller  computer 
envisioned  for  the  model.  As  a  result,  a  substantial  portion  of  soft¬ 
ware  development  was  straightforward  implementing  of  the  computer  and 
the  peripheral  equipment  to  perform  designated  functions,  and  a  minor 
portion  was  devoted  to  simulating  missing  equipment.  Whenever  modifi¬ 
cations  in  computer  operations,  programming,  and  operator  procedures 
were  made,  the  prime  objective  was  to  endow  the  simulation  with  only 
those  man-machine  interfaces  and  procedures  that  were  prescribed  for  the 
model . 


^Commercial  designations  are  given  only  in  the  interest  of  precise  de¬ 
scription;  their  mention  does  not  constitute  indorsement  by  BESRL  or 
by  the  Army. 
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systems  and  in  providing  systems  developers  with  information  useful  in  design  specifi¬ 
cations  for  future  systems. 


|\r\  fMM  I  A  "to  REBLACE*  do  form  i«t».  i 
UV  I  m7h|4  /O  0B60LBTS  BON  ARMY  WBE. 


JAN  •«,  WHICH  IB 


-  55  - 


Unclassified 
■•curity  Claaairication 


14 


Unclassified 
icurity  Classification 


KIV  WOKOt 

Interpreter  techniques 
Image  interpretation  displays 
♦Intelligence  systems 
♦Image  systems 
♦Surveillance  facility 
♦Summary  -  surveillance  research  program 
Information  assimilation 
Information  processing 
Information  storage  -  retrieval 
Laboratory  facilities 
Aerial  surveillance 
Feedback  techniques 

Computer-compatible  interpreter  procedures 
TIIF 

Image  systems  models--task--functions 
♦Systems  evaluation- -Measurement- -Methodology 
Simulated  computer  procedures 
Field  command  and  control  facilities 
Interpreter  team  performance 
Interpreter  performance  measures 
♦Data  bank  utilization 
Advanced  computerized  systems 
♦Man/computer  decision  processes 
♦Computer  aided  instruction  (CAl) 
Photointerpretation  systems- -techniques 


